什么是可扩展性？

What Is Scalability?

直观上，可扩展性是一个非常简单的概念。如果我们向维基百科询问定义，它会告诉我们：“可扩展性是系统的属性，通过向系统添加资源来处理不断增长的工作量。” 我们都知道如何扩展高速公路系统——我们添加更多车道，以便它可以处理更多数量的车辆。我最喜欢的一些人知道如何扩大啤酒生产规模——他们在酿造容器的数量和大小、执行和管理酿造过程的员工数量以及可以填充新鲜啤酒的小桶数量方面增加了更多的产能。美味的啤酒。想想任何物理系统——交通系统、机场、建筑物中的电梯——我们如何增加容量是非常明显的。

Intuitively, scalability is a pretty straightforward concept. If we ask Wikipedia for a definition, it tells us, “Scalability is the property of a system to handle a growing amount of work by adding resources to the system.” We all know how we scale a highway system—we add more traffic lanes so it can handle a greater number of vehicles. Some of my favorite people know how to scale beer production—they add more capacity in terms of the number and size of brewing vessels, the number of staff to perform and manage the brewing process, and the number of kegs they can fill with fresh, tasty brews. Think of any physical system—a transit system, an airport, elevators in a building—and how we increase capacity is pretty obvious.

与物理系统不同，软件系统有些无定形。它们不是你可以指着、看到、触摸、感觉的东西，也不能从外部观察中了解它的内部行为。软件系统是一个数字工件。从本质上讲，组成可执行代码和数据的 1 和 0 流任何人都很难区分。那么，可扩展性对于软件系统来说意味着什么呢？

Unlike physical systems, software systems are somewhat amorphous. They are not something you can point at, see, touch, feel, and get a sense of how it behaves internally from external observation. A software system is a digital artifact. At its core, the stream of 1s and 0s that make up executable code and data are hard for anyone to tell apart. So, what does scalability mean in terms of a software system?

简而言之，无需陷入定义之争，可扩展性定义了软件系统处理其操作某些方面的增长的能力。操作维度的示例有：

Put very simply, and without getting into definition wars, scalability defines a software system’s capability to handle growth in some dimension of its operations. Examples of operational dimensions are:

• 系统可以处理的同时用户或外部（例如传感器）请求的数量

• The number of simultaneous user or external (e.g., sensor) requests a system can process

• 系统可以有效处理和管理的数据量

• The amount of data a system can effectively process and manage

• 通过预测分析可以从系统存储的数据中获得的价值

• The value that can be derived from the data a system stores through predictive analytics

• 随着请求负载的增长，能够保持稳定、一致的响应时间

• The ability to maintain a stable, consistent response time as the request load grows

例如，假设一家大型连锁超市正在迅速开设新店，并增加每家商店中自助结账亭的数量。这就要求超市核心软件系统能够完成以下功能：

For example, imagine a major supermarket chain is rapidly opening new stores and increasing the number of self-checkout kiosks in every store. This requires the core supermarket software systems to perform the following functions:

• 处理因项目扫描而增加的数量，而不减少响应时间。为了让顾客满意，对商品扫描的即时响应是必要的。

• Handle increased volume from item scanning without decreased response time. Instantaneous responses to item scans are necessary to keep customers happy.

• 处理和存储因销售额增加而产生的大量数据。库存管理、会计、规划以及可能的许多其他功能都需要这些数据。

• Process and store the greater data volumes generated from increased sales. This data is needed for inventory management, accounting, planning, and likely many other functions.

• 从每个商店、地区和国家获得“实时”（例如每小时）销售数据摘要，并与历史趋势进行比较。这些趋势数据可以帮助突出显示该地区的异常事件（意外的天气条件、活动中的大量人群等），并帮助受影响的商店快速做出响应。

• Derive “real-time” (e.g., hourly) sales data summaries from each store, region, and country and compare to historical trends. This trend data can help highlight unusual events in regions (unexpected weather conditions, large crowds at events, etc.) and help affected stores to quickly respond.

• 改进库存订购预测子系统，以便能够随着商店和客户数量的增长正确预测销售情况（从而预测库存重新订购的需要）。

• Evolve the stock ordering prediction subsystem to be able to correctly anticipate sales (and hence the need for stock reordering) as the number of stores and customers grow.

这些维度实际上是系统的可扩展性要求。如果连锁超市在一年多时间里开设了 100 家新店，销售额增长了 400 倍（其中一些新店规模很大！），那么软件系统需要进行扩展，以提供必要的处理能力，使超市能够高效运营。如果系统无法扩展，当客户不满意时我们可能会失去销售额。我们可能持有不会很快售出的库存，从而增加成本。我们可能会错过因应当地情况而推出特别产品来增加销量的机会。所有这些因素都会降低客户满意度和利润。没有一个对商业有利。

These dimensions are effectively the scalability requirements of the system. If, over a year, the supermarket chain opens 100 new stores and grows sales by 400 times (some of the new stores are big!), then the software system needs to scale to provide the necessary processing capacity to enable the supermarket to operate efficiently. If the systems don’t scale, we could lose sales when customers become unhappy. We might hold stock that will not be sold quickly, increasing costs. We might miss opportunities to increase sales by responding to local circumstances with special offerings. All these factors reduce customer satisfaction and profits. None are good for business.

因此，成功的扩展对于我们想象中的超市的业务增长至关重要，实际上也是许多现代互联网应用程序的命脉。但对于大多数企业和政府系统来说，可扩展性并不是开发和部署早期阶段的首要质量要求。增强可用性和实用性的新功能成为我们开发周期的驱动力。只要正常负载下性能足够，我们就会不断添加面向用户的功能来提升系统的商业价值。事实上，在有明确的需求之前引入我将在本书中描述的一些复杂的分布式技术实际上可能对项目有害，因为额外的复杂性会导致开发惯性。

Successfully scaling is therefore crucial for our imaginary supermarket’s business growth, and likewise is in fact the lifeblood of many modern internet applications. But for most business and government systems, scalability is not a primary quality requirement in the early stages of development and deployment. New features to enhance usability and utility become the drivers of our development cycles. As long as performance is adequate under normal loads, we keep adding user-facing features to enhance the system’s business value. In fact, introducing some of the sophisticated distributed technologies I’ll describe in this book before there is a clear requirement can actually be deleterious to a project, with the additional complexity causing development inertia.

尽管如此，系统发展到增强性能和可扩展性成为紧迫问题甚至生存问题的情况并不罕见。有吸引力的功能和高实用性孕育了成功，这带来了更多的请求需要处理和更多的数据需要管理。这通常预示着一个转折点，在轻负载下有意义的设计决策突然变成了技术债务。¹外部触发事件通常会导致这些临界点：请参阅 2020 年 3 月/4 月媒体，了解有关政府失业和超市在线订购网站因冠状病毒大流行导致的需求而崩溃的许多报道。

Still, it’s not uncommon for systems to evolve into a state where enhanced performance and scalability become a matter of urgency, or even survival. Attractive features and high utility breed success, which brings more requests to handle and more data to manage. This often heralds a tipping point, wherein design decisions that made sense under light loads suddenly become technical debt.¹ External trigger events often cause these tipping points: look in the March/April 2020 media for the many reports of government unemployment and supermarket online ordering sites crashing under demand caused by the coronavirus pandemic.

通过增加资源来增加系统在某些方面的容量称为纵向扩展或横向扩展——稍后我将探讨它们之间的区别。此外，与物理系统不同，能够缩小系统容量以降低成本通常同样重要。

Increasing a system’s capacity in some dimension by increasing resources is called scaling up or scaling out—I’ll explore the difference between these later. In addition, unlike physical systems, it is often equally important to be able to scale down the capacity of a system to reduce costs.

典型的例子是 Netflix，它需要处理可预测的区域昼夜负载。简而言之，在任何地理区域，晚上 9 点观看 Netflix 的人数都比早上 5 点观看的人数要多得多，这使得 Netflix 能够在负载较低时减少其处理资源。这节省了运行亚马逊云中使用的处理节点的成本，以及降低数据中心功耗等对社会有价值的事情。将其与高速公路进行比较。晚上路上车少的时候，我们不会收回车道（维修除外）。道路的全部通行能力可供少数驾驶员随心所欲地行驶。在软件系统中，我们可以在几秒钟内扩展和收缩我们的处理能力，以满足瞬时负载。与物理系统相比，

The canonical example of this is Netflix, which has a predictable regional diurnal load that it needs to process. Simply, a lot more people are watching Netflix in any geographical region at 9 p.m. than are at 5 a.m. This enables Netflix to reduce its processing resources during times of lower load. This saves the cost of running the processing nodes that are used in the Amazon cloud, as well as societally worthy things such as reducing data center power consumption. Compare this to a highway. At night when few cars are on the road, we don’t retract lanes (except to make repairs). The full road capacity is available for the few drivers to go as fast as they like. In software systems, we can expand and contract our processing capacity in a matter of seconds to meet instantaneous load. Compared to physical systems, the strategies we deploy are vastly different.

关于软件系统的可扩展性，还有很多事情需要考虑，但让我们在检查 2021 年左右的一些当代软件系统的规模后回到这些问题。

There’s a lot more to consider about scalability in software systems, but let’s come back to these issues after examining the scale of some contemporary software systems circa 2021.

2000 年代初的系统规模示例

Examples of System Scale in the Early 2000s

在这场技术游戏中展望未来总是充满危险。2008 年我写道：

Looking ahead in this technology game is always fraught with danger. In 2008 I wrote:

“虽然 PB 数据集和千兆位数据流是当今数据密集型应用程序的前沿，但毫无疑问，从现在开始 10 年后，我们将深深地回忆起这种规模的问题，并担心即将出现的百亿亿次应用程序所带来的困难。” ²

“While petabyte datasets and gigabit data streams are today’s frontiers for data-intensive applications, no doubt 10 years from now we’ll fondly reminisce about problems of this scale and be worrying about the difficulties that looming exascale applications are posing.”²

合理的情绪，这是事实，但是百亿亿级呢？这在当今世界几乎是司空见惯的事情。Google在 2014 年报告了数 EB 的Gmail ，而现在，所有 Google 服务都管理 1 YO 字节或更多吗？我不知道。我什至不确定我是否知道 yottabyte 是什么！谷歌不会告诉我们他们的存储情况，但我不会打赌它。同样，Amazon 在各种 AWS 数据存储中为其客户存储了多少数据？例如，对于所有支持的客户端应用程序，DynamoDB 每秒总共处理多少个请求？这些事情想得太久，你的头就会爆炸。

Reasonable sentiments, it is true, but exascale? That’s almost commonplace in today’s world. Google reported multiple exabytes of Gmail in 2014, and by now, do all Google services manage a yottabyte or more? I don’t know. I’m not even sure I know what a yottabyte is! Google won’t tell us about their storage, but I wouldn’t bet against it. Similarly, how much data does Amazon store in the various AWS data stores for their clients? And how many requests does, say, DynamoDB process per second, collectively, for all supported client applications? Think about these things for too long and your head will explode.

主要互联网公司的技术博客是一个重要的信息来源，有时可以洞察当代的运营规模。还有一些分析互联网流量的网站可以很好地说明流量。让我们举几个时间点的例子来说明我们今天所知道的一些事情。请记住，在一四年内这些将看起来几乎古怪：

A great source of information that sometimes gives insights into contemporary operational scales are the major internet companies’ technical blogs. There are also websites analyzing internet traffic that are highly illustrative of traffic volumes. Let’s take a couple of point-in-time examples to illustrate a few things we do know today. Bear in mind these will look almost quaint in a year or four:

• Facebook 的工程博客介绍了Scribe，这是他们每小时收集、聚合和交付 PB 级日志数据的解决方案，具有低延迟和高吞吐量。Facebook 的计算基础设施由数百万台机器组成，每台机器都会生成日志文件，捕获与系统和应用程序运行状况相关的重要事件。处理这些日志文件（例如来自 Web 服务器的日志文件）可以让开发团队深入了解其应用程序的行为和性能，并支持故障查找。Scribe 是一种自定义缓冲排队解决方案，可以以每秒几 TB 的速率传输来自服务器的日志，并将其传送到下游分析和数据仓库系统。我的朋友们，这是很多数据！

• Facebook’s engineering blog describes Scribe, their solution for collecting, aggregating, and delivering petabytes of log data per hour, with low latency and high throughput. Facebook’s computing infrastructure comprises millions of machines, each of which generates log files that capture important events relating to system and application health. Processing these log files, for example from a web server, can give development teams insights into their application’s behavior and performance, and support faultfinding. Scribe is a custom buffered queuing solution that can transport logs from servers at a rate of several terabytes per second and deliver them to downstream analysis and data warehousing systems. That, my friends, is a lot of data!

• 您可以在Internet Live Stats中查看众多服务的实时 Internet 流量。仔细研究一下，你会发现一些惊人的统计数据；例如，Google 每天处理约 35 亿个搜索请求，Instagram 用户每天上传约 6500 万张照片，网站数量约为 17 亿个。这是一个有趣的网站，有很多信息。请注意，这些数据不是真实的，而是基于多个数据源的统计分析的估计值。

• You can see live internet traffic for numerous services at Internet Live Stats. Dig around and you’ll find some staggering statistics; for example, Google handles around 3.5 billion search requests per day, Instagram users upload about 65 million photos per day, and there are something like 1.7 billion websites. It is a fun site with lots of information. Note that the data is not real, but rather estimates based on statistical analyses of multiple data sources.

• 2016年，谷歌发表了一篇论文，描述了其代码库的特点。在报告的众多令人震惊的事实中，有一个事实是“该存储库包含 86 TB 的数据，其中包括 900 万个独特源文件中的大约 20 亿行代码。” 请记住，这是 2016 年^。3

• In 2016, Google published a paper describing the characteristics of its codebase. Among the many startling facts reported is the fact that “The repository contains 86 TBs of data, including approximately two billion lines of code in nine million unique source files.” Remember, this was 2016.³

尽管如此，主要互联网网站提供的服务规模的真实、具体数据仍然处于商业机密之中。幸运的是，我们可以通过一家科技公司的年度使用报告深入了解互联网规模处理的请求和数据量。但要小心，因为它来自 Pornhub。⁴您可以在此处浏览2019 年以来极其详细的使用统计数据。这是对大规模系统能力的一次令人着迷的一瞥。

Still, real, concrete data on the scale of the services provided by major internet sites remain shrouded in commercial-in-confidence secrecy. Luckily, we can get some deep insights into the request and data volumes handled at internet scale through the annual usage report from one tech company. Beware though, as it is from Pornhub.⁴ You can browse their incredibly detailed usage statistics from 2019 here. It’s a fascinating glimpse into the capabilities of massive-scale systems.

我们是怎么来到这里的？系统发展简史

How Did We Get Here? A Brief History of System Growth

我相信许多读者很难相信在互联网搜索、YouTube 和社交媒体出现之前就已经存在文明生活了。事实上，第一个视频上传到 YouTube是在 2005 年。是的，连我都难以置信。那么，让我们简要回顾一下我们是如何达到当今系统规模的。以下是一些值得注意的历史里程碑：

I am sure many readers will have trouble believing there was civilized life before internet searching, YouTube, and social media. In fact, the first video upload to YouTube occurred in 2005. Yep, it is hard even for me to believe. So, let’s take a brief look back in time at how we arrived at the scale of today’s systems. Below are some historical milestones of note:

20世纪80年代

1980s

一个由分时大型机和小型机主导的时代。PC 出现于 20 世纪 80 年代初，但很少联网。到 20 世纪 80 年代末，开发实验室、大学和（越来越多的）企业都拥有了电子邮件和原始互联网资源。

An age dominated by time-shared mainframes and minicomputers. PCs emerged in the early 1980s but were rarely networked. By the end of the 1980s, development labs, universities, and (increasingly) businesses had email and access to primitive internet resources.

1990–95

1990–95

网络变得更加普遍，为使用 HTTP/HTML 技术创建万维网 (WWW) 创建了一个成熟的环境，该技术由 Tim Berners-Lee在 20 世纪 80 年代在 CERN 首创。到 1995 年，网站的数量还很少，但像 Yahoo! 这样的公司已经播下了未来的种子。1994 年，亚马逊和 eBay 于 1995 年。

Networks became more pervasive, creating an environment ripe for the creation of the World Wide Web (WWW) with HTTP/HTML technology that had been pioneered at CERN by Tim Berners-Lee during the 1980s. By 1995, the number of websites was tiny, but the seeds of the future were planted with companies like Yahoo! in 1994 and Amazon and eBay in 1995.

1996–2000

1996–2000

网站数量从1万个左右增长到1000万个，这是一个真正的爆炸性增长时期。网络带宽和访问也快速增长。亚马逊、eBay、谷歌和雅虎等公司 我们为我们今天所了解和使用的高度可扩展的系统开创了许多设计原则和先进技术的早期版本。日常企业争先恐后地利用电子商务提供的新机会，这使得系统可扩展性变得突出，正如侧边栏“规模如何影响业务系统”中所解释的那样。

The number of websites grew from around 10,000 to 10 million, a truly explosive growth period. Networking bandwidth and access also grew rapidly. Companies like Amazon, eBay, Google, and Yahoo! were pioneering many of the design principles and early versions of advanced technologies for highly scalable systems that we know and use today. Everyday businesses rushed to exploit the new opportunities that e-business offered, and this brought system scalability to prominence, as explained in the sidebar “How Scale Impacted Business Systems”.

2000年–2006年

2000–2006

在此期间，网站数量从大约 1000 万个增长到 8000 万个，新的服务和商业模式也出现了。2005年，YouTube推出。2006 年 Facebook 向公众开放。同年，2004 年低调起步的亚马逊网络服务 (AWS) 重新推出 S3 和 EC2 服务。

The number of websites grew from around 10 million to 80 million during this period, and new service and business models emerged. In 2005, YouTube was launched. 2006 saw Facebook become available to the public. In the same year, Amazon Web Services (AWS), which had low-key beginnings in 2004, relaunched with its S3 and EC2 services.

2007 年至今

2007–today

我们现在生活在一个拥有大约 20 亿个网站的世界，其中大约 20% 是活跃的。互联网用户约有40 亿。由 AWS、谷歌云平台 (GCP) 和 Microsoft Azure 等公共云运营商运营的大型数据中心，以及无数私有数据中心（例如Twitter 的运营基础设施）分布在全球各地。云托管着数百万个应用程序，工程师使用复杂的云管理门户来配置和操作其计算和数据存储系统。强大的云服务使我们只需点击几下鼠标就可以构建、部署和扩展我们的系统。公司所要做的就是在月底支付云提供商的账单。

We now live in a world with around 2 billion websites, of which about 20% are active. There are something like 4 billion internet users. Huge data centers operated by public cloud operators like AWS, Google Cloud Platform (GCP), and Microsoft Azure, along with a myriad of private data centers, for example, Twitter’s operational infrastructure, are scattered around the planet. Clouds host millions of applications, with engineers provisioning and operating their computational and data storage systems using sophisticated cloud management portals. Powerful cloud services make it possible for us to build, deploy, and scale our systems literally with a few clicks of a mouse. All companies have to do is pay their cloud provider bill at the end of the month.

这就是本书所针对的世界。在这个世界中，我们的应用程序需要利用构建可扩展系统的关键原则并利用高度可扩展的基础设施平台。请记住，在现代应用程序中，执行的大部分代码不是由您的组织编写的。它是容器、数据库、消息传递系统以及您通过 API 调用和构建指令组合到应用程序中的其他组件的一部分。这使得这些组件的选择和使用至少与您自己的业务逻辑的设计和开发一样重要。它们是不容易改变的架构决策。

This is the world that this book targets. A world where our applications need to exploit the key principles for building scalable systems and leverage highly scalable infrastructure platforms. Bear in mind, in modern applications, most of the code executed is not written by your organization. It is part of the containers, databases, messaging systems, and other components that you compose into your application through API calls and build directives. This makes the selection and use of these components at least as important as the design and development of your own business logic. They are architectural decisions that are not easy to change.

可扩展性基本设计原则

Scalability Basic Design Principles

扩展系统的基本目标是增加其在某些特定于应用程序的维度上的容量。一个常见的维度是增加系统在给定时间段内可以处理的请求数量。这称为系统的吞吐量。让我们用一个类比来探讨我们可用于扩展系统和增加吞吐量的两个基本原则：复制和优化。

The basic aim of scaling a system is to increase its capacity in some application-specific dimension. A common dimension is increasing the number of requests that a system can process in a given time period. This is known as the system’s throughput. Let’s use an analogy to explore two basic principles we have available to us for scaling our systems and increasing throughput: replication and optimization.

1932 年，世界标志性工程奇迹之一悉尼海港大桥落成启用。现在，可以相当安全地假设 2021 年的交通量会比 1932 年有所增加。如果您在过去 30 年中曾在高峰时段开车过这座桥，那么您就会知道它的通行能力每天都会大大超出。那么我们如何提高桥梁等物理基础设施的吞吐量呢？

In 1932, one of the world’s iconic wonders of engineering, the Sydney Harbour Bridge, was opened. Now, it is a fairly safe assumption that traffic volumes in 2021 are somewhat higher than in 1932. If by any chance you have driven over the bridge at peak hour in the last 30 years, then you know that its capacity is exceeded considerably every day. So how do we increase throughput on physical infrastructures such as bridges?

这个问题在 20 世纪 80 年代的悉尼变得非常突出，当时人们意识到必须增加海港过境的容量。解决方案是不太具有标志性的悉尼海港隧道，它基本上遵循海港下方的相同路线。这提供了四个额外的交通车道，从而增加了大约三分之一的港口过境能力。在不远的奥克兰，他们的海港大桥也存在通行能力问题，因为它建于 1959 年，只有四车道。本质上，他们采取了与悉尼相同的解决方案，即增加运力。但他们并没有建造隧道，而是巧妙地通过用名字搞笑的“日本夹子”扩大桥梁，使车道数量增加了一倍。，加宽了桥的两侧。

This issue became very prominent in Sydney in the 1980s, when it was realized that the capacity of the harbor crossing had to be increased. The solution was the rather less iconic Sydney Harbour Tunnel, which essentially follows the same route underneath the harbor. This provides four additional lanes of traffic and hence added roughly one-third more capacity to harbor crossings. In not-too-far-away Auckland, their harbor bridge also had a capacity problem as it was built in 1959 with only four lanes. In essence, they adopted the same solution as Sydney, namely, to increase capacity. But rather than build a tunnel, they ingeniously doubled the number of lanes by expanding the bridge with the hilariously named “Nippon clip-ons”, which widened the bridge on each side.

这些例子说明了我们在软件系统中提高容量的第一个策略。我们基本上复制了软件处理资源，以提供更多的能力来处理请求，从而提高吞吐量，如图1-1所示。这些复制的处理资源类似于桥梁上的车道，为到达的请求流提供基本上独立的处理路径。

These examples illustrate the first strategy we have in software systems to increase capacity. We basically replicate the software processing resources to provide more capacity to handle requests and thus increase throughput, as shown in Figure 1-1. These replicated processing resources are analogous to the traffic lanes on bridges, providing a mostly independent processing pathway for a stream of arriving requests.

幸运的是，在基于云的软件系统中，只需点击鼠标即可实现复制，我们可以有效地复制我们的处理资源数千次。在这方面我们比桥梁建造者容易得多。尽管如此，我们仍然需要注意复制资源，以缓解真正的瓶颈。向未不堪重负的处理路径添加容量会增加不必要的成本，而不会提供可扩展性优势。

Luckily, in cloud-based software systems, replication can be achieved at the click of a mouse, and we can effectively replicate our processing resources thousands of times. We have it a lot easier than bridge builders in that respect. Still, we need to take care to replicate resources in order to alleviate real bottlenecks. Adding capacity to processing paths that are not overwhelmed will add needless costs without providing scalability benefit.

图 1-1。通过复制增加容量

第二种可扩展性策略也可以通过我们的桥接示例来说明。在悉尼，一些细心的人发现，早上有更多的车辆从北向南过桥，而下午我们看到相反的情况。因此，一个聪明的解决方案被设计出来——在早上将更多的车道分配到高需求方向，并在下午的某个时间进行切换。这有效地增加了桥梁的容量，而无需分配任何新资源——我们优化了已有的可用资源。

The second strategy for scalability can also be illustrated with our bridge example. In Sydney, some observant person realized that in the mornings a lot more vehicles cross the bridge from north to south, and in the afternoon we see the reverse pattern. A smart solution was therefore devised—allocate more of the lanes to the high-demand direction in the morning, and sometime in the afternoon, switch this around. This effectively increased the capacity of the bridge without allocating any new resources—we optimized the resources we already had available.

我们可以在软件中遵循同样的方法来扩展我们的系统。如果我们能够通过使用更高效的算法、在数据库中添加额外的索引来加速查询、甚至用更快的编程语言重写我们的服务器来以某种方式优化我们的处理，那么我们就可以在不增加资源的情况下增加我们的容量。典型的例子是 Facebook为 PHP 创建的 HipHop（现已停产），它通过将 PHP 代码编译为 C++，将 Facebook 网页生成的速度提高了六倍。

We can follow this same approach in software to scale our systems. If we can somehow optimize our processing by using more efficient algorithms, adding extra indexes in our databases to speed up queries, or even rewriting our server in a faster programming language, we can increase our capacity without increasing our resources. The canonical example of this is Facebook’s creation of (the now discontinued) HipHop for PHP, which increased the speed of Facebook’s web page generation by up to six times by compiling PHP code to C++.

我将在本书中重新审视这两个设计原则，即复制和优化。您将看到，由于我们正在构建分布式系统，因此采用这些原则会产生许多复杂的影响。分布式系统具有使构建可扩展系统变得有趣的属性，在这种情况下，它既有积极的含义，也有消极的含义。

I’ll revisit these two design principles—namely replication and optimization—throughout this book. You will see that there are many complex implications of adopting these principles, arising from the fact that we are building distributed systems. Distributed systems have properties that make building scalable systems interesting, which in this context has both positive and negative connotations.

可扩展性和成本

Scalability and Costs

让我们用一个简单的假设示例来检查可扩展性和成本之间的关系。假设我们有一个基于 Web（例如 Web 服务器和数据库）的系统，可以为 100 个并发请求的负载提供服务，平均响应时间为 1 秒。我们收到一项业务需求，需要扩展该系统，以在相同的响应时间下处理 1,000 个并发请求。在不进行任何更改的情况下，对该系统进行简单的负载测试即可显示如图 1-2 （左）所示的性能。随着请求负载的增加，我们看到平均响应时间随着预计负载稳定增长到 10 秒。显然这不能满足我们当前部署配置的要求。系统无法扩展。

Let’s take a trivial hypothetical example to examine the relationship between scalability and costs. Assume we have a web-based (e.g., web server and database) system that can service a load of 100 concurrent requests with a mean response time of 1 second. We get a business requirement to scale up this system to handle 1,000 concurrent requests with the same response time. Without making any changes, a simple load test of this system reveals the performance shown in Figure 1-2 (left). As the request load increases, we see the mean response time steadily grow to 10 seconds with the projected load. Clearly this does not satisfy our requirements in its current deployment configuration. The system doesn’t scale.

图 1-2。扩展应用程序；左侧表示不可扩展性能，右侧表示可扩展性能

为了达到所需的性能，需要一些工程努力。图 1-2（右）显示了修改此工作后系统的性能。现在，它可以在 1,000 个并发请求下提供指定的响应时间。因此，我们已经成功扩展了系统。晚会时间！

Some engineering effort is needed in order to achieve the required performance. Figure 1-2 (right) shows the system’s performance after this effort has been modified. It now provides the specified response time with 1,000 concurrent requests. And so, we have successfully scaled the system. Party time!

然而，一个重大问题迫在眉睫。即，需要多少努力和资源才能实现这一绩效？也许这只是在更强大的（虚拟）机器上运行 Web 服务器的情况。在云上执行此类重新配置最多可能需要 30 分钟。稍微复杂一点的是重新配置系统以运行 Web 服务器的多个实例以增加容量。同样，这应该是一个简单、低成本的应用程序配置更改，无需更改代码。这些将是极好的结果。

A major question looms, however. Namely, how much effort and resources were required to achieve this performance? Perhaps it was simply a case of running the web server on a more powerful (virtual) machine. Performing such reprovisioning on a cloud might take 30 minutes at most. Slightly more complex would be reconfiguring the system to run multiple instances of the web server to increase capacity. Again, this should be a simple, low-cost configuration change for the application, with no code changes needed. These would be excellent outcomes.

然而，扩展系统并不总是那么容易。造成这种情况的原因多种多样，但以下是一些可能性：

However, scaling a system isn’t always so easy. The reasons for this are many and varied, but here are some possibilities:

• 数据库的响应速度降低，每秒 1,000 个请求，需要升级到新机器。

• The database becomes less responsive with 1,000 requests per second, requiring an upgrade to a new machine.

• Web 服务器动态生成大量内容，这减少了负载下的响应时间。一种可能的解决方案是更改代码以更有效地生成内容，从而减少每个请求的处理时间。

• The web server generates a lot of content dynamically and this reduces response time under load. A possible solution is to alter the code to more efficiently generate the content, thus reducing processing time per request.

• 当许多请求尝试同时访问和更新相同的记录时，请求负载会在数据库中创建热点。这需要重新设计架构并随后重新加载数据库，以及对数据访问层的代码进行更改。

• The request load creates hotspots in the database when many requests try to access and update the same records simultaneously. This requires a schema redesign and subsequent reloading of the database, as well as code changes to the data access layer.

• 所选择的 Web 服务器框架强调开发的简易性而不是可扩展性。它强制执行的模型意味着代码根本无法扩展以满足请求的负载要求，并且需要完全重写。使用另一个框架？甚至使用另一种编程语言？

• The web server framework that was selected emphasized ease of development over scalability. The model it enforces means that the code simply cannot be scaled to meet the requested load requirements, and a complete rewrite is required. Use another framework? Use another programming language even?

还有无数其他潜在原因，但希望这些说明了当我们从可能性 (1) 转向可能性 (4) 时可能需要付出越来越多的努力。

There’s a myriad of other potential causes, but hopefully these illustrate the increasing effort that might be required as we move from possibility (1) to possibility (4).

现在让我们假设选项 (1)，即升级数据库服务器，需要 15 个小时的工作量以及每月 1000 美元的额外云成本才能获得更强大的服务器。这并不是非常昂贵。我们假设选项 (4)，即重写 Web 应用程序层，由于实施新语言（例如，Java 而不是 Ruby），需要 10,000 小时的开发时间。选项（2）和（3）介于选项（1）和（4）之间。10,000 小时的开发成本非常巨大。更糟糕的是，在开发过程中，应用程序可能会因为无法满足客户请求的负载而失去市场份额，从而失去金钱。此类情况可能会导致系统和业务失败。

Now let’s assume option (1), upgrading the database server, requires 15 hours of effort and a thousand dollars in extra cloud costs per month for a more powerful server. This is not prohibitively expensive. And let’s assume option (4), a rewrite of the web application layer, requires 10,000 hours of development due to implementing a new language (e.g., Java instead of Ruby). Options (2) and (3) fall somewhere in between options (1) and (4). The cost of 10,000 hours of development is seriously significant. Even worse, while the development is underway, the application may be losing market share and hence money due to its inability to satisfy client requests’ loads. These kinds of situations can cause systems and businesses to fail.

这个简单的场景说明了资源和工作成本的维度如何与可扩展性密不可分地联系在一起。如果系统本质上不是按规模设计的，那么增加其容量以满足需求的下游成本和资源可能会很大。对于某些应用程序，例如HealthCare.gov，需要承担这些（超过 20 亿美元）成本，并对系统进行修改以最终满足业务需求。对于其他国家，例如俄勒冈州的医疗保健交易所，无法以低成本快速扩展可能会造成昂贵的损失（俄勒冈州的情况为 3.03 亿美元）。

This simple scenario illustrates how the dimensions of resource and effort costs are inextricably tied to scalability. If a system is not designed intrinsically to scale, then the downstream costs and resources of increasing its capacity to meet requirements may be massive. For some applications, such as HealthCare.gov, these (more than $2 billion) costs are borne and the system is modified to eventually meet business needs. For others, such as Oregon’s health care exchange, an inability to scale rapidly at low cost can be an expensive ($303 million, in Oregon’s case) death knell.

我们绝不会想到有人会尝试将郊区住宅的容量扩大为 50 层的办公楼。这座房子没有建筑、材料和地基，如果不完全拆除和重建，这甚至是一个遥远的可能性。同样，我们不应该期望不采用可扩展架构、机制和技术的软件系统能够快速发展以满足更大的容量需求。规模化的基础需要从一开始就建立起来，并认识到组件会随着时间的推移而发展。通过采用促进可扩展性的设计和开发原则，我们可以更快、更便宜地扩展系统，以满足快速增长的需求。我将在本书的第二部分中解释这些原则。

We would never expect someone would attempt to scale up the capacity of a suburban home to become a 50-floor office building. The home doesn’t have the architecture, materials, and foundations for this to be even a remote possibility without being completely demolished and rebuilt. Similarly, we shouldn’t expect software systems that do not employ scalable architectures, mechanisms, and technologies to be quickly evolved to meet greater capacity needs. The foundations of scale need to be built in from the beginning, with the recognition that the components will evolve over time. By employing design and development principles that promote scalability, we can more rapidly and cheaply scale up systems to meet rapidly growing demands. I’ll explain these principles in Part II of this book.

可以指数级扩展而成本线性增长的软件系统被称为超大规模系统，我将其定义如下：“超可扩展系统在计算和存储能力方面表现出指数增长，同时在构建、运营所需的资源成本方面表现出线性增长。” 、支持和发展所需的软件和硬件资源。” 您可以在本文中阅读有关超大规模系统的更多信息。

Software systems that can be scaled exponentially while costs grow linearly are known as hyperscale systems, which I define as follows: “Hyper scalable systems exhibit exponential growth in computational and storage capabilities while exhibiting linear growth rates in the costs of resources required to build, operate, support, and evolve the required software and hardware resources.” You can read more about hyperscale systems in this article.

可扩展性和架构的权衡

Scalability and Architecture Trade-Offs

可扩展性只是众多质量属性或非功能性需求之一，它们是软件架构学科的通用语言。软件架构持久的复杂性之一是质量属性权衡的必要性。基本上，有利于一种质量属性的设计可能会对其他质量属性产生负面或正面影响。例如，当我们的服务中发生某些事件时，我们可能希望写入日志消息，以便我们可以进行取证并支持代码的调试。但是，我们需要小心捕获的事件数量，因为日志记录会带来开销并对性能和成本产生负面影响。

Scalability is just one of the many quality attributes, or nonfunctional requirements, that are the lingua franca of the discipline of software architecture. One of the enduring complexities of software architecture is the necessity of quality attribute trade-offs. Basically, a design that favors one quality attribute may negatively or positively affect others. For example, we may want to write log messages when certain events occur in our services so we can do forensics and support debugging of our code. We need to be careful, however, how many events we capture, because logging introduces overheads and negatively affects performance and cost.

经验丰富的软件架构师不断谨慎行事，精心设计以满足高优先级的质量属性，同时最大限度地减少对其他质量属性的负面影响。

Experienced software architects constantly tread a fine line, crafting their designs to satisfy high-priority quality attributes, while minimizing the negative effects on other quality attributes.

可扩展性也不例外。当我们关注系统的可扩展能力时，我们必须仔细考虑我们的设计如何影响其他非常理想的属性，例如性能、可用​​性、安全性以及经常被忽视的可管理性。我将在以下各节中简要讨论其中一些固有的权衡。

Scalability is no different. When we point the spotlight at the ability of a system to scale, we have to carefully consider how our design influences other highly desirable properties such as performance, availability, security, and the oft overlooked manageability. I’ll briefly discuss some of these inherent trade-offs in the following sections.

总结和延伸阅读

Summary and Further Reading

快速且经济高效地扩展应用程序的能力应该是当代面向互联网的应用程序的软件架构的定义品质。我们有两种基本方法来实现可扩展性，即增加系统容量（通常通过复制）和系统组件的性能优化。

The ability to scale an application quickly and cost-effectively should be a defining quality of the software architecture of contemporary internet-facing applications. We have two basic ways to achieve scalability, namely increasing system capacity, typically through replication, and performance optimization of system components.

与任何软件架构质量属性一样，可扩展性无法单独实现。它不可避免地涉及复杂的权衡，需要根据应用程序的要求进行调整。我将在本书的其余部分讨论这些基本的权衡，从下一章开始，我将描述实现可扩展性的具体架构方法。

Like any software architecture quality attribute, scalability cannot be achieved in isolation. It inevitably involves complex trade-offs that need to be tuned to an application’s requirements. I’ll be discussing these fundamental trade-offs throughout the remainder of this book, starting in the next chapter when I describe concrete architecture approaches to achieve scalability.

基本系统架构

Basic System Architecture

事实上，所有大规模系统都是从小规模开始的，并因其成功而不断发展。从 Ruby on Rails、Django 或类似框架等开发框架入手是常见且明智的做法，它可以促进快速开发，使系统快速启动并运行。图 2-1显示了一个典型的非常简单的“入门”系统软件架构，它与快速开发框架非常相似。这包括客户端层、应用程序服务层和数据库层。如果您使用 Rails 或同等产品，您还可以获得一个框架，该框架硬连线用于 Web 应用程序处理的模型-视图-控制器 (MVC) 模式以及生成 SQL 查询的对象关系映射器 (ORM)。

Virtually all massive-scale systems start off small and grow due to their success. It’s common, and sensible, to start with a development framework such as Ruby on Rails, Django, or equivalent, which promotes rapid development to get a system quickly up and running. A typical very simple software architecture for “starter” systems, which closely resembles what you get with rapid development frameworks, is shown in Figure 2-1. This comprises a client tier, application service tier, and a database tier. If you use Rails or equivalent, you also get a framework which hardwires a model–view–controller (MVC) pattern for web application processing and an object–relational mapper (ORM) that generates SQL queries.

图 2-1。基本的多层分布式系统架构

通过这种架构，用户可以从移动应用程序或 Web 浏览器向应用程序提交请求。互联网的魔力（参见第 3 章）将这些请求传递到在某些企业或商业云数据中心托管的计算机上运行的应用程序服务。通信使用标准应用程序级网络协议，通常是 HTTP。

With this architecture, users submit requests to the application from their mobile app or web browser. The magic of internet networking (see Chapter 3) delivers these requests to the application service which is running on a machine hosted in some corporate or commercial cloud data center. Communications uses a standard application-level network protocol, typically HTTP.

应用程序服务运行支持客户端用来发送 HTTP 请求的 API 的代码。收到请求后，服务将执行与所请求的 API 关联的代码。在此过程中，它可能会读取或写入数据库或其他外部系统，具体取决于 API 的语义。请求完成后，服务会将结果发送到客户端以显示在其应用程序或浏览器中。

The application service runs code supporting an API that clients use to send HTTP requests. Upon receipt of a request, the service executes the code associated with the requested API. In the process, it may read from or write to a database or some other external system, depending on the semantics of the API. When the request is complete, the service sends the results to the client to display in their app or browser.

许多（如果不是大多数）系统在概念上看起来与此完全相同。应用程序服务代码利用服务器执行环境，该环境允许同时处理来自多个用户的多个请求。有无数种应用程序服务器技术（例如，Java EE 和 Spring Framework for Java、Flask for Python）在此场景中广泛使用。

Many, if not most systems conceptually look exactly like this. The application service code exploits a server execution environment that enables multiple requests from multiple users to be processed simultaneously. There’s a myriad of these application server technologies—for example, Java EE and the Spring Framework for Java, Flask for Python—that are widely used in this scenario.

这种方法导致了通常所说的整体架构。¹随着应用程序的功能变得更加丰富，单体应用的复杂性往往会增加。所有 API 处理程序都内置到同一服务器代码体中。这最终会导致快速修改和测试变得困难，并且由于所有 API 实现都在同一应用程序服务中运行，因此执行足迹可能会变得极其繁重。

This approach leads to what is generally known as a monolithic architecture.¹ Monoliths tend to grow in complexity as the application becomes more feature-rich. All API handlers are built into the same server code body. This eventually makes it hard to modify and test rapidly, and the execution footprint can become extremely heavyweight as all the API implementations run in the same application service.

不过，如果请求负载保持相对较低，这种应用程序架构就足够了。该服务能够以始终如一的低延迟处理请求。但如果请求负载持续增长，这意味着延迟将会增加，因为服务没有足够的 CPU/内存容量来处理并发请求量，因此请求将需要更长的时间来处理。在这种情况下，我们的单台服务器就超载了，成为了瓶颈。

Still, if request loads stay relatively low, this application architecture can suffice. The service has the capacity to process requests with consistently low latency. But if request loads keep growing, this means latencies will increase as the service has insufficient CPU/memory capacity for the concurrent request volume and therefore requests will take longer to process. In these circumstances, our single server is overloaded and has become a bottleneck.

在这种情况下，第一个扩展策略通常是“扩展”应用服务硬件。例如，如果您的应用程序在 AWS 上运行，您可以将服务器从具有四个（虚拟）CPU 和 16 GB 内存的普通 t3.xlarge 实例升级到 t3.2xlarge 实例，这将使可用的 CPU 和内存数量加倍对于应用程序。²

In this case, the first strategy for scaling is usually to “scale up” the application service hardware. For example, if your application is running on AWS, you might upgrade your server from a modest t3.xlarge instance with four (virtual) CPUs and 16 GB of memory to a t3.2xlarge instance, which doubles the number of CPUs and memory available for the application.²

扩大规模很简单。它使许多现实世界的应用程序能够支持更大的工作负载。显然，硬件成本更高，但这对你来说是可扩展的。

Scaling up is simple. It gets many real-world applications a long way to supporting larger workloads. It obviously costs more money for hardware, but that’s scaling for you.

然而，对于许多应用程序来说，无论有多少个 CPU 和多少内存，负载都会不可避免地增长到淹没单个服务器节点的水平。这时候你就需要一个新的策略——即横向扩展，或者水平扩展，我在第一章中谈到过。

It’s inevitable, however, that for many applications the load will grow to a level which will swamp a single server node, no matter how many CPUs and how much memory you have. That’s when you need a new strategy—namely, scaling out, or horizontal scaling, which I touched on in Chapter 1.

向外扩展

Scale Out

横向扩展依赖于在架构中复制服务并在多个服务器节点上运行多个副本的能力。来自客户端的请求分布在各个副本上，因此理论上，如果我们有 N 个副本和 R 个请求，则每个服务器节点都会处理 R/N 个请求。这个简单的策略增加了应用程序的容量，从而提高了可扩展性。

Scaling out relies on the ability to replicate a service in the architecture and run multiple copies on multiple server nodes. Requests from clients are distributed across the replicas so that in theory, if we have N replicas and R requests, each server node processes R/N requests. This simple strategy increases an application’s capacity and hence scalability.

要成功扩展应用程序，您的设计中需要两个基本元素。如图 2-2所示，它们是：

To successfully scale out an application, you need two fundamental elements in your design. As illustrated in Figure 2-2, these are:

负载均衡器

A load balancer

所有用户请求都发送到负载均衡器，负载均衡器选择服务副本目标来处理请求。选择目标服务有多种策略，所有策略的核心目标都是保持每个资源同等繁忙。负载均衡器还将服务的响应转发回客户端。大多数负载平衡器属于一类称为反向代理的互联网组件。它们控制客户端请求对服务器资源的访问。作为中介，反向代理为请求添加额外的网络跃点；它们需要极低的延迟，以最大限度地减少它们引入的开销。有许多现成的负载平衡解决方案以及云提供商特定的解决方案，我将在第 5 章中更详细地介绍这些解决方案的一般特征。

All user requests are sent to a load balancer, which chooses a service replica target to process the request. Various strategies exist for choosing a target service, all with the core aim of keeping each resource equally busy. The load balancer also relays the responses from the service back to the client. Most load balancers belong to a class of internet components known as reverse proxies. These control access to server resources for client requests. As an intermediary, reverse proxies add an extra network hop for a request; they need to be extremely low latency to minimize the overheads they introduce. There are many off-the-shelf load balancing solutions as well as cloud provider–specific ones, and I’ll cover the general characteristics of these in much more detail in Chapter 5.

无状态服务

Stateless services

为了使负载均衡有效并均匀地共享请求，负载均衡器必须能够自由地将来自同一客户端的连续请求发送到不同的服务实例进行处理。这意味着服务中的 API 实现不得保留与单个客户端会话相关的任何知识或状态。当用户访问应用程序时，服务会创建一个用户会话，并在内部管理一个唯一的会话，以识别用户交互的顺序并跟踪会话状态。会话状态的一个典型示例是购物车。为了有效地使用负载均衡器，表示用户购物车当前内容的数据必须存储在某个地方（通常是数据存储），以便任何服务副本在收到作为用户会话一部分的请求时都可以访问此状态。在图 2-2，这被标记为“会话存储”。

For load balancing to be effective and share requests evenly, the load balancer must be free to send consecutive requests from the same client to different service instances for processing. This means the API implementations in the services must retain no knowledge, or state, associated with an individual client’s session. When a user accesses an application, a user session is created by the service and a unique session is managed internally to identify the sequence of user interactions and track session state. A classic example of session state is a shopping cart. To use a load balancer effectively, the data representing the current contents of a user’s cart must be stored somewhere—typically a data store—such that any service replica can access this state when it receives a request as part of a user session. In Figure 2-2, this is labeled as a “Session store.”

横向扩展很有吸引力，因为从理论上讲，您可以不断添加新的（虚拟）硬件和服务来处理增加的请求负载并保持请求延迟一致且较低。一旦发现延迟上升，就部署另一个服务器实例。这不需要使用无状态服务更改代码，因此相对便宜 - 您只需为部署的硬件付费。

Scaling out is attractive as, in theory, you can keep adding new (virtual) hardware and services to handle increased request loads and keep request latencies consistent and low. As soon as you see latencies rising, you deploy another server instance. This requires no code changes with stateless services and is relatively cheap as a result—you just pay for the hardware you deploy.

横向扩展还有另一个极具吸引力的功能。如果其中一项服务出现故障，它正在处理的请求将会丢失。但由于失败的服务不管理会话状态，因此客户端可以简单地重新发出这些请求并将其发送到另一个服务实例进行处理。这意味着应用程序能够适应服务软件和硬件的故障，从而增强应用程序的可用性。

Scaling out has another highly attractive feature. If one of the services fails, the requests it is processing will be lost. But as the failed service manages no session state, these requests can be simply reissued by the client and sent to another service instance for processing. This means the application is resilient to failures in the service software and hardware, thus enhancing the application’s availability.

不幸的是，与任何工程解决方案一样，简单的横向扩展也有局限性。当您添加新的服务实例时，请求处理能力会增长，甚至可能无限增长。然而，在某些阶段，现实将会残酷，单个数据库提供低延迟查询响应的能力将会减弱。缓慢的查询将意味着客户端的响应时间更长。如果请求到达的速度始终快于处理的速度，某些系统组件将变得过载并因资源耗尽而失败，并且客户端将看到异常和请求超时。从本质上讲，您的数据库成为您必须消除的瓶颈，以便进一步扩展您的应用程序。

Unfortunately, as with any engineering solution, simple scaling out has limits. As you add new service instances, the request processing capacity grows, potentially infinitely. At some stage, however, reality will bite and the capability of your single database to provide low-latency query responses will diminish. Slow queries will mean longer response times for clients. If requests keep arriving faster than they are being processed, some system components will become overloaded and fail due to resource exhaustion, and clients will see exceptions and request timeouts. Essentially, your database becomes a bottleneck that you must engineer away in order to scale your application further.

图 2-2。横向扩展架构

通过缓存扩展数据库

Scaling the Database with Caching

通过增加数据库服务器中的 CPU、内存和磁盘数量来进行扩展对于扩展系统大有帮助。例如，在撰写本文时，GCP 可以在 db-n1-highmem-96 节点上配置 SQL 数据库，该节点具有 96 个虚拟 CPU (vCPU)、624 GB 内存、30 TB 磁盘，并且可以支持 4,000 个连接。每年的费用在 6,000 美元到 16,000 美元之间，这对我来说听起来很划算！纵向扩展是一种常见的数据库可扩展性策略。

Scaling up by increasing the number of CPUs, memory, and disks in a database server can go a long way to scaling a system. For example, at the time of writing, GCP can provision a SQL database on a db-n1-highmem-96 node, which has 96 virtual CPUs (vCPUs), 624 GB of memory, 30 TB of disk, and can support 4,000 connections. This will cost somewhere between $6K and $16K per year, which sounds like a good deal to me! Scaling up is a common database scalability strategy.

大型数据库需要高技能数据库管理员的持续关注和关注，以保持其调整和快速运行。这项工作涉及很多技巧，例如查询调优、磁盘分区、索引、节点上缓存等等，因此数据库管理员是您想要非常友善的有价值的人。它们可以使您的应用程序服务具有高度响应能力。

Large databases need constant care and attention from highly skilled database administrators to keep them tuned and running fast. There’s a lot of wizardry in this job—e.g., query tuning, disk partitioning, indexing, on-node caching, and so on—and hence database administrators are valuable people you want to be very nice to. They can make your application services highly responsive.

与扩展相结合，一种非常有效的方法是尽可能少地从服务中查询数据库。这可以通过在横向扩展服务层中使用分布式缓存来实现。将最近检索和经常访问的数据库结果缓存存储在内存中，以便可以快速检索它们，而不会给数据库带来负担。例如，下一个小时的天气预报不会改变，但可能会被数百或数千个客户端查询。发布后，您可以使用缓存来存储预测。所有客户端请求都将从缓存中读取，直到预测过期。

In conjunction with scaling up, a highly effective approach is querying the database as infrequently as possible from your services. This can be achieved by employing distributed caching in the scaled-out service tier. Caching stores recently retrieved and commonly accessed database results in memory so they can be quickly retrieved without placing a burden on the database. For example, the weather forecast for the next hour won’t change, but may be queried by hundreds or thousands of clients. You can use a cache to store the forecast once it is issued. All client requests will read from the cache until the forecast expires.

对于经常读取且很少更改的数据，可以修改处理逻辑以首先检查分布式缓存，例如 Redis或memcached存储。这些缓存技术本质上是具有非常简单的 API 的分布式键值存储。该方案如图2-3所示。请注意，图 2-2中的会话存储已消失。这是因为您可以使用通用分布式缓存来存储会话标识符和应用程序数据。

For data that is frequently read and changes rarely, your processing logic can be modified to first check a distributed cache, such as a Redis or memcached store. These cache technologies are essentially distributed key-value stores with very simple APIs. This scheme is illustrated in Figure 2-3. Note that the session store from Figure 2-2 has disappeared. This is because you can use a general-purpose distributed cache to store session identifiers along with application data.

图 2-3。引入分布式缓存

访问缓存需要从您的服务进行远程调用。如果您需要的数据在缓存中，那么在快速网络上，您可以期待亚毫秒级的缓存读取。这比查询共享数据库实例的成本要低得多，并且也不需要查询来争夺通常稀缺的数据库连接。

Accessing the cache requires a remote call from your service. If the data you need is in the cache, on a fast network you can expect submillisecond cache reads. This is far less expensive than querying the shared database instance, and also doesn’t require a query to contend for typically scarce database connections.

引入缓存层还需要修改您的处理逻辑以检查缓存的数据。如果缓存中没有您想要的内容，您的代码仍然必须查询数据库并将结果加载到缓存中并将其返回给调用者。您还需要决定何时删除或使缓存的结果失效——您的操作过程取决于数据的性质（例如，天气预报自然过期）以及您的应用程序对服务过时（也称为过时）的容忍度——给客户的结果。

Introducing a caching layer also requires your processing logic to be modified to check for cached data. If what you want is not in the cache, your code must still query the database and load the results into the cache as well as return it to the caller. You also need to decide when to remove, or invalidate, cached results—your course of action depends on the nature of your data (e.g., weather forecasts expire naturally) and your application’s tolerance to serving out-of-date—also known as stale—results to clients.

精心设计的缓存方案对于扩展系统来说非常宝贵。缓存非常适合很少更改且经常访问的数据，例如库存目录、事件信息和联系人数据。如果您可以处理很大比例（例如 80% 或更多）的缓存读取请求，那么您可以有效地在数据库中购买额外的容量，因为它们永远不会看到很大比例的请求。

A well-designed caching scheme can be invaluable in scaling a system. Caching works great for data that rarely changes and is accessed frequently, such as inventory catalogs, event information, and contact data. If you can handle a large percentage, say, 80% or more, of read requests from your cache, then you effectively buy extra capacity at your databases as they never see a large proportion of requests.

尽管如此，许多系统需要快速访问 TB 级和更大的数据存储，这使得单个数据库实际上令人望而却步。在这些系统中，需要分布式数据库。

Still, many systems need to rapidly access terabytes and larger data stores that make a single database effectively prohibitive. In these systems, a distributed database is needed.

分发数据库

Distributing the Database

到 2022 年，分布式数据库技术的数量可能超出您的想象。这是一个复杂的领域，我将在第三部分中广泛介绍这一领域。一般来说，有两大类：

There are more distributed database technologies around in 2022 than you probably want to imagine. It’s a complex area, and one I’ll cover extensively in Part III. In very general terms, there are two major categories:

分布式 SQL 存储

Distributed SQL stores

这些使组织能够通过将数据存储在多个数据库引擎副本查询的多个磁盘上，相对无缝地扩展其 SQL 数据库。这些多个引擎在逻辑上对应用程序显示为单个数据库，从而最大限度地减少代码更改。还有一类“天生分布式”SQL 数据库，通常称为 NewSQL 存储，属于此类。

These enable organizations to scale out their SQL database relatively seamlessly by storing the data across multiple disks that are queried by multiple database engine replicas. These multiple engines logically appear to the application as a single database, hence minimizing code changes. There is also a class of “born distributed” SQL databases that are commonly known as NewSQL stores that fit in this category.

分布式所谓的“NoSQL”存储（来自一系列供应商）

Distributed so-called “NoSQL” stores (from a whole array of vendors)

这些产品使用各种数据模型和查询语言在运行数据库引擎的多个节点之间分发数据，每个节点都有自己的本地连接存储。同样，数据的位置对于应用程序来说是透明的，并且通常由使用数据库键上的散列函数的数据模型的设计来控制。该类别中的领先产品是 Cassandra、MongoDB 和 Neo4j。

These products use a variety of data models and query languages to distribute data across multiple nodes running the database engine, each with their own locally attached storage. Again, the location of the data is transparent to the application, and typically controlled by the design of the data model using hashing functions on database keys. Leading products in this category are Cassandra, MongoDB, and Neo4j.

图 2-4显示了我们的架构如何整合分布式数据库。随着数据量的增长，分布式数据库可以增加存储节点的数量。添加（或删除）节点时，会重新平衡所有节点上管理的数据，以确保平等地利用每个节点的处理和存储容量。

Figure 2-4 shows how our architecture incorporates a distributed database. As the data volumes grow, a distributed database can increase the number of storage nodes. As nodes are added (or removed), the data managed across all nodes is rebalanced to attempt to ensure the processing and storage capacity of each node is equally utilized.

图 2-4。使用分布式数据库扩展数据层

分布式数据库还可以提高可用性。它们支持复制每个数据存储节点，因此如果一个节点出现故障或由于网络问题而无法访问，则可以使用另一份数据副本。用于复制的模型以及这些模型所需的权衡（剧透警告：一致性）将在后面的章节中介绍。

Distributed databases also promote availability. They support replicating each data storage node so if one fails or cannot be accessed due to network problems, another copy of the data is available. The models utilized for replication and the trade-offs these require (spoiler alert: consistency) are covered in later chapters.

如果您使用的是主要的云提供商，则您的数据层还有两种部署选择。您可以部署自己的虚拟资源并构建、配置和管理自己的分布式数据库服务器。或者，您可以使用云托管数据库。后者简化了与管理、监控和扩展数据库相关的管理工作，因为其中许多任务基本上成为您选择的云提供商的责任。与往常一样，没有免费的午餐原则适用。无论您选择哪种方式，您总是要付费的。

If you are utilizing a major cloud provider, there are also two deployment choices for your data tier. You can deploy your own virtual resources and build, configure, and administer your own distributed database servers. Alternatively, you can utilize cloud-hosted databases. The latter simplifies the administrative effort associated with managing, monitoring, and scaling the database, as many of these tasks essentially become the responsibility of the cloud provider you choose. As usual, the no free lunch principle applies. You always pay, whichever approach you choose.

多个处理层

Multiple Processing Tiers

您需要扩展的任何实际系统都将具有许多不同的服务，这些服务通过交互来处理请求。例如，访问Amazon.com上的网页可能需要调用超过 100 个不同的服务才能将响应返回给用户。³

Any realistic system that you need to scale will have many different services that interact to process a request. For example, accessing a web page on Amazon.com can require in excess of 100 different services being called before a response is returned to the user.³

我在本章中阐述的无状态、负载平衡、缓存架构的优点在于可以扩展核心设计原则并构建多层应用程序。在满足请求时，服务可以调用一个或多个依赖的服务，这些服务依次被复制和负载平衡。一个简单的例子如图2-5所示。服务如何交互以及应用程序如何确保依赖服务的快速响应存在许多细微差别。同样，我将在后面的章节中详细介绍这些内容。

The beauty of the stateless, load-balanced, cached architecture I am elaborating in this chapter is that it’s possible to extend the core design principles and build a multitiered application. In fulfilling a request, a service can call one or more dependent services, which in turn are replicated and load-balanced. A simple example is shown in Figure 2-5. There are many nuances in how the services interact, and how applications ensure rapid responses from dependent services. Again, I’ll cover these in detail in later chapters.

图 2-5。多级扩展处理能力

这种设计还促进在架构中的每一层拥有不同的负载平衡服务。例如，图 2-6说明了两个复制的面向 Internet 的服务，它们都使用提供数据库访问的核心服务。每个服务都是负载平衡的，并使用缓存来提供高性能和可用性。这种设计通常用于为 Web 客户端和移动客户端提供服务，每个客户端都可以根据它们所经历的负载独立扩展。它通常称为前端后端 (BFF) 模式。⁴

This design also promotes having different, load-balanced services at each tier in the architecture. For example, Figure 2-6 illustrates two replicated internet-facing services that both utilized a core service that provides database access. Each service is load balanced and employs caching to provide high performance and availability. This design is often used to provide a service for web clients and a service for mobile clients, each of which can be scaled independently based on the load they experience. It’s commonly called the Backend for Frontend (BFF) pattern.⁴

图 2-6。具有多种服务的可扩展架构

此外，通过将应用程序分解为多个独立的服务，您可以根据服务需求扩展每个服务。例如，如果您发现移动用户的请求量不断增加，而 Web 用户的请求量不断减少，则可以为每项服务配置不同数量的实例来满足需求。这是将单体应用程序重构为多个独立服务的主要优点，这些服务可以单独构建、测试、部署和扩展。我将在第 9 章中探讨基于此类服务（称为微服务）设计系统的一些主要问题。

In addition, by breaking the application into multiple independent services, you can scale each based on the service demand. If, for example, you see an increasing volume of requests from mobile users and decreasing volumes from web users, it’s possible to provision different numbers of instances for each service to satisfy demand. This is a major advantage of refactoring monolithic applications into multiple independent services, which can be separately built, tested, deployed, and scaled. I’ll explore some of the major issues in designing systems based on such services, known as microservices, in Chapter 9.

提高响应能力

Increasing Responsiveness

大多数客户端应用程序请求都期望得到响应。用户可能希望查看给定产品类别的所有拍卖项目或查看给定位置可出售的房地产。在这些示例中，客户端发送请求并等待收到响应。发送请求到收到结果之间的这个时间间隔就是请求的响应时间。您可以通过使用缓存和预先计算的响应来减少响应时间，但许多请求仍然会导致数据库访问。

Most client application requests expect a response. A user might want to see all auction items for a given product category or see the real estate that is available for sale in a given location. In these examples, the client sends a request and waits until a response is received. This time interval between sending the request and receiving the result is the response time of the request. You can decrease response times by using caching and precalculated responses, but many requests will still result in database accesses.

对于更新应用程序中的数据的请求也存在类似的情况。如果用户在下订单之前更新其送货地址，则必须保留新的送货地址，以便用户可以在点击“购买”按钮之前确认该地址。这种情况下的响应时间包括数据库写入的时间，该时间由用户收到的响应确认。

A similar scenario exists for requests that update data in an application. If a user updates their delivery address immediately prior to placing an order, the new delivery address must be persisted so that the user can confirm the address before they hit the “purchase” button. The response time in this case includes the time for the database write, which is confirmed by the response the user receives.

然而，某些更新请求可以成功响应，而无需将数据完全保留在数据库中。例如，滑雪者和滑雪板爱好者会熟悉缆车票扫描系统，该系统会检查您是否拥有当天乘坐缆车的有效通行证。他们还记录您乘坐的电梯、乘坐的时间等等。然后，滑雪爱好者/单板滑雪爱好者可以使用度假村的移动应用程序查看他们一天乘坐的缆车次数。

Some update requests, however, can be successfully responded to without fully persisting the data in a database. For example, the skiers and snowboarders among you will be familiar with lift ticket scanning systems that check you have a valid pass to ride the lifts that day. They also record which lifts you take, the time you get on, and so on. Nerdy skiers/snowboarders can then use the resort’s mobile app to see how many lifts they ride in a day.

当一个人等待乘电梯时，扫描仪设备会使用 RFID（射频识别）芯片读取器来验证通行证。然后，有关骑手、缆车和时间的信息通过互联网发送到滑雪胜地运营的数据采集服务。电梯乘客不必等待这种情况发生，因为响应时间可能会减慢电梯装载过程。电梯乘客也不期望他们可以立即使用他们的应用程序来确保捕获这些数据。他们只是坐上电梯，和朋友聊聊，然后计划下一次跑步。

As a person waits to get on a lift, a scanner device validates the pass using an RFID (radio-frequency identification) chip reader. The information about the rider, lift, and time are then sent over the internet to a data capture service operated by the ski resort. The lift rider doesn’t have to wait for this to occur, as the response time could slow down the lift-loading process. There’s also no expectation from the lift rider that they can instantly use their app to ensure this data has been captured. They just get on the lift, talk smack with their friends, and plan their next run.

服务实现可以利用这种类型的场景来提高响应能力。有关事件的数据被发送到服务，该服务确认接收并同时将数据存储在远程队列中，以便随后写入数据库。分布式排队平台可用于将数据从一个服务可靠地发送到另一个服务，通常但并不总是以先进先出 (FIFO) 方式。

Service implementations can exploit this type of scenario to improve responsiveness. The data about the event is sent to the service, which acknowledges receipt and concurrently stores the data in a remote queue for subsequent writing to the database. Distributed queueing platforms can be used to reliably send data from one service to another, typically but not always in a first-in, first-out (FIFO) manner.

将消息写入队列通常比写入数据库快得多，这使得请求能够更快地成功确认。部署另一个服务来从队列中读取消息并将数据写入数据库。当滑雪者检查他们的缆车时（可能是三个小时或三天后），数据已成功保存在数据库中。

Writing a message to a queue is typically much faster than writing to a database, and this enables the request to be successfully acknowledged much more quickly. Another service is deployed to read messages from the queue and write the data to the database. When a skier checks their lift rides—maybe three hours or three days later—the data has been persisted successfully in the database.

实现该方法的基本架构如图2-7所示。

The basic architecture to implement this approach is illustrated in Figure 2-7.

图 2-7。通过排队提高响应能力

只要不立即需要写入操作的结果，应用程序就可以使用此方法来提高响应能力，从而提高可扩展性。应用程序可以利用许多排队技术，我将在第 7 章中讨论这些技术的运作方式。这些排队平台都提供异步通信。生产者服务写入充当临时存储的队列，而另一个消费者服务从队列中删除消息并对存储滑雪者缆车乘坐详细信息的数据库进行必要的更新。

Whenever the results of a write operation are not immediately needed, an application can use this approach to improve responsiveness and, as a result, scalability. Many queueing technologies exist that applications can utilize, and I’ll discuss how these operate in Chapter 7. These queueing platforms all provide asynchronous communications. A producer service writes to the queue, which acts as temporary storage, while another consumer service removes messages from the queue and makes the necessary updates to, in our example, a database that stores skier lift ride details.

关键是数据最终会被持久化。最终通常意味着最多几秒钟，但采用这种设计的用例应该能够适应更长的延迟，而不影响用户体验。

The key is that the data eventually gets persisted. Eventually typically means a few seconds at most but use cases that employ this design should be resilient to longer delays without impacting the user experience.

系统和硬件可扩展性

Systems and Hardware Scalability

如果服务和数据存储在不足的硬件上运行，即使是最精心设计的软件架构和代码也会在可扩展性方面受到限制。通常部署在可扩展系统中的开源和商业平台旨在利用 CPU 内核、内存和磁盘等额外硬件资源。这是实现您所需的性能和可扩展性与尽可能降低成本之间的平衡行为。

Even the most carefully crafted software architecture and code will be limited in terms of scalability if the services and data stores run on inadequate hardware. The open source and commercial platforms that are commonly deployed in scalable systems are designed to utilize additional hardware resources in terms of CPU cores, memory, and disks. It’s a balancing act between achieving the performance and scalability you require, and keeping your costs as low as possible.

也就是说，在某些情况下，升级 CPU 核心数量和可用内存并不能为您带来更多的可扩展性。例如，如果代码是单线程的，那么在具有更多内核的节点上运行它不会提高性能。它在任何时候都只使用一个核心。其余的根本就没有使用。如果多线程代码包含许多序列化部分，则一次只能有一个线程核心继续执行，以确保结果正确。这种现象由阿姆达尔定律描述。这为我们提供了一种根据串行执行的代码量来计算添加更多 CPU 内核时代码的理论加速的方法。

That said, there are some cases where upgrading the number of CPU cores and available memory is not going to buy you more scalability. For example, if code is single threaded, running it on a node with more cores is not going to improve performance. It’ll just use one core at any time. The rest are simply not used. If multithreaded code contains many serialized sections, only one threaded core can proceed at a time to ensure the results are correct. This phenomenon is described by Amdahl’s law. This gives us a way to calculate the theoretical acceleration of code when adding more CPU cores based on the amount of code that executes serially.

阿姆达尔定律的两个数据点是：

Two data points from Amdahl’s law are:

• 如果只有 5% 的代码串行执行，其余代码并行执行，那么添加超过 2,048 个核心基本上没有任何效果。

• If only 5% of a code executes serially, the rest in parallel, adding more than 2,048 cores has essentially no effect.

• 如果 50% 的代码串行执行，其余代码并行执行，那么添加超过 8 个核心基本上没有任何效果。

• If 50% of a code executes serially, the rest in parallel, adding more than 8 cores has essentially no effect.

这说明了为什么高效的多线程代码对于实现可扩展性至关重要。如果您的代码没有像线程那样运行高度独立的任务，那么即使有钱也买不到可扩展性。这就是为什么我在第 4 章专门讨论多线程主题——它是构建可扩展分布式系统的核心知识组件。

This demonstrates why efficient, multithreaded code is essential to achieving scalability. If your code is not running as highly independent tasks implemented as threads, then not even money will buy you scalability. That’s why I devote Chapter 4 to the topic of multithreading—it’s a core knowledge component for building scalable distributed systems.

为了说明升级硬件的效果，图 2-8显示了当数据库部署在更强大（且更昂贵）的硬件上时，基准系统的吞吐量如何提高。⁵该基准测试采用 Java 服务，该服务接受来自负载生成客户端的请求、查询数据库并将结果返回给客户端。客户端、服务和数据库运行在部署在AWS云中相同区域的不同硬件资源上。

To illustrate the effect of upgrading hardware, Figure 2-8 shows how the throughput of a benchmark system improves as the database is deployed on more powerful (and expensive) hardware.⁵ The benchmark employs a Java service that accepts requests from a load generating client, queries a database, and returns the results to the client. The client, service, and database run on different hardware resources deployed in the same regions in the AWS cloud.

图 2-8。扩展数据库服务器的示例

在测试中，并发请求数从 32 个增加到 256 个（x轴），每条线代表AWS EC2 关系数据库服务 (RDS) 上不同硬件配置的系统吞吐量（y轴）。图表底部列出了不同的配置，左侧最弱，右侧最强大。每个客户端通过 HTTP 同步发送固定数量的请求，在接收一个请求的结果和发送下一个请求之间没有暂停。因此，这会对服务器施加较高的请求负载。

In the tests, the number of concurrent requests grows from 32 to 256 (x-axis) and each line represents the system throughput (y-axis) for a different hardware configuration on the AWS EC2’s Relational Database Service (RDS). The different configurations are listed at the bottom of the chart, with the least powerful on the left and most powerful on the right. Each client sends a fixed number of requests synchronously over HTTP, with no pause between receiving results from one request and sending the next. This consequently exerts a high request load on the server.

从这张图表中，可以得出一些简单的观察结果：

From this chart, it’s possible to make some straightforward observations:

• 一般来说，为数据库选择的硬件越强大，吞吐量就越高。那很好。

• In general, the more powerful the hardware selected for the database, the higher the throughput. That is good.

• db.t2.xlarge 和 db.t2.2xlarge 实例在吞吐量方面的差异很小。这可能是因为服务层正在成为瓶颈，或者我们的数据库模型和查询没有利用 db.t2.2xlarge RDS 实例的额外资源。不管怎样——花更多的钱，却没有任何好处。

• The difference between the db.t2.xlarge and db.t2.2xlarge instances in terms of throughput is minimal. This could be because the service tier is becoming a bottleneck, or our database model and queries are not exploiting the additional resources of the db.t2.2xlarge RDS instance. Regardless—more bucks, no bang.

• 在请求负载增加到 256 个并发客户端之前，两个功能最弱的实例表现得相当好。这两个实例的吞吐量下降表明它们已经过载，如果请求负载增加，情况只会变得更糟。

• The two least powerful instances perform pretty well until the request load is increased to 256 concurrent clients. The dip in throughput for these two instances indicates they are overloaded and things will only get worse if the request load increases.

希望这个简单的例子能够说明为什么需要谨慎地通过简单的硬件升级来进行扩展。添加更多硬件总是会增加成本，但可能并不总能带来您期望的性能改进。运行简单的实验并进行测量对于评估硬件升级的效果至关重要。它为您提供可靠的数据来指导您的设计并向利益相关者证明成本的合理性。

Hopefully, this simple example illustrates why scaling through simple upgrading of hardware needs to be approached carefully. Adding more hardware always increases costs, but may not always give the performance improvement you expect. Running simple experiments and taking measurements is essential for assessing the effects of hardware upgrades. It gives you solid data to guide your design and justify costs to stakeholders.

总结和延伸阅读

Summary and Further Reading

在本章中，我简要介绍了可用于将系统扩展为通信服务和分布式数据库集合的主要方法。许多细节都被忽略了，而且您无疑已经意识到，在软件系统中，细节决定成败。因此，后续章节将逐步开始探讨这些细节，从第 3 章中每个人都应该了解的分布式系统的一些基本特征开始。

In this chapter I’ve provided a whirlwind tour of the major approaches you can utilize to scale out a system as a collection of communicating services and distributed databases. Much detail has been brushed over, and as you have no doubt realized—in software systems the devil is in the detail. Subsequent chapters will therefore progressively start to explore these details, starting with some fundamental characteristics of distributed systems in Chapter 3 that everyone should be aware of.

本章回避的另一个领域是软件架构主题。我在实现应用程序业务逻辑和数据库访问的体系结构中使用术语服务来表示分布式组件。这些服务是独立部署的进程，使用 HTTP 等远程通信机制进行通信。在架构方面，这些服务与面向服务的架构 (SOA) 模式（一种用于构建分布式系统的既定架构方法）中的服务最接近。这种方法的更现代的演变围绕着微服务。这些往往是更具凝聚力的封装服务，可促进持续开发和部署。

Another area this chapter has skirted around is the subject of software architecture. I’ve used the term services for distributed components in an architecture that implement application business logic and database access. These services are independently deployed processes that communicate using remote communications mechanisms such as HTTP. In architectural terms, these services are most closely mirrored by those in the service-oriented architecture (SOA) pattern, an established architectural approach for building distributed systems. A more modern evolution of this approach revolves around microservices. These tend to be more cohesive, encapsulated services that promote continuous development and deployment.

如果您想更深入地讨论这些问题以及一般的软件架构概念，那么 Mark Richards 和 Neal Ford 的书《Fundamentals of Software Architecture: An Engineering Approach》（O'Reilly，2020）是一个很好的地方开始。

If you’d like a much more in-depth discussion of these issues, and software architecture concepts in general, then Mark Richards’ and Neal Ford’s book Fundamentals of Software Architecture: An Engineering Approach (O’Reilly, 2020) is an excellent place to start.

最后，有一类大数据软件架构可以解决非常大的数据集合中出现的一些问题。最突出的问题之一是数据再处理。当由于代码或业务规则更改而需要重新分析已存储和分析的数据时，就会发生这种情况。这种重新处理可能是由于软件修复或引入可以从原始数据中获得更多见解的新算法而发生的。在O'Reilly Radar 博客上的Jay Krepps 2014 年文章“质疑 Lambda 架构”中，对 Lambda 和 Kappa 架构进行了很好的讨论，这两种架构在该领域都很突出。

Finally, there’s a class of big data software architectures that address some of the issues that come to the fore with very large data collections. One of the most prominent is data reprocessing. This occurs when data that has already been stored and analyzed needs to be reanalyzed due to code or business rule changes. This reprocessing may occur due to software fixes, or the introduction of new algorithms that can derive more insights from the original raw data. There’s a good discussion of the Lambda and Kappa architectures, both of which are prominent in this space, in Jay Krepps’ 2014 article “Questioning the Lambda Architecture” on the O’Reilly Radar blog.

通讯基础知识

Communications Basics

每个分布式系统都有通过网络进行通信的软件组件。如果移动银行应用程序请求用户当前的银行帐户余额，则会发生以下（非常简单的）通信序列：

Every distributed system has software components that communicate over a network. If a mobile banking app requests the user’s current bank account balance, a (very simplified) sequence of communications occurs along the lines of:

1. 移动银行应用程序通过蜂窝网络向银行发送请求，以检索用户的银行余额。

2. The mobile banking app sends a request over the cellular network addressed to the bank to retrieve the user’s bank balance.

3. 该请求通过互联网路由到银行网络服务器所在的位置。

4. The request is routed across the internet to where the bank’s web servers are located.

5. 银行的 Web 服务器对请求进行身份验证（确认它源自假定的用户）并向数据库服务器发送请求以获取帐户余额。

6. The bank’s web server authenticates the request (confirms that it originated from the supposed user) and sends a request to a database server for the account balance.

7. 数据库服务器从磁盘读取账户余额并将其返回给Web服务器。

8. The database server reads the account balance from disk and returns it to the web server.

9. 网络服务器通过回复消息发送余额到应用程序，该消息通过互联网和蜂窝网络路由，直到余额神奇地出现在移动设备的屏幕上。

10. The web server sends the balance in a reply message addressed to the app, which is routed over the internet and the cellular network until the balance magically appears on the screen of the mobile device.

当您阅读上面的内容时，听起来似乎很简单，但实际上，这一系列内容背后隐藏着巨大的复杂性。通讯。让我们在以下部分中研究其中的一些复杂性。

It almost sounds simple when you read the above, but in reality, there’s a huge amount of complexity hidden beneath this sequence of communications. Let’s examine some of these complexities in the following sections.

通讯硬件

Communications Hardware

上面的银行余额请求示例将不可避免地遍历多种不同的网络技术，并且设备。全球互联网是一台异构机器，由不同类型的网络通信通道和设备组成，每秒通过网络将数百万条消息传送到预定目的地。

The bank balance request example above will inevitably traverse multiple different networking technologies and devices. The global internet is a heterogeneous machine, comprising different types of network communications channels and devices that shuttle many millions of messages per second across networks to their intended destinations.

存在不同类型的通信渠道。最明显的分类是有线与无线。对于每个类别，都有多种网络传输硬件技术可以将比特从一台机器传输到另一台机器。每种技术都有不同的特征，我们通常关心的是速度和范围。

Different types of communications channels exist. The most obvious categorization is wired versus wireless. For each category there are multiple network transmission hardware technologies that can ship bits from one machine to another. Each technology has different characteristics, and the ones we typically care about are speed and range.

对于物理有线网络，两个最常见类型是局域网 (LAN) 和广域网 (WAN)。LAN 是可以连接“建筑规模”设备的网络，能够在少量（例如 1-2）公里内传输数据。数据中心中的现代 LAN 可以传输每秒 10 到 100 吉比特 (Gbps) 的数据。这是已知的作为网络的带宽或容量。使用现代 LAN 技术，通过 LAN 传输消息所需的时间（即网络延迟）为亚毫秒级。

For physically wired networks, the two most common types are local area networks (LANs) and wide area networks (WANs). LANs are networks that can connect devices at “building scale,” being able to transmit data over a small number (e.g., 1–2) of kilometers. Contemporary LANs in data centers can transport between 10 and 100 gigabits per second (Gbps). This is known as the network’s bandwidth, or capacity. The time taken to transmit a message across a LAN—the network’s latency—is submillisecond with modern LAN technologies.

广域网是遍布全球的网络，构成了我们统称为互联网的网络。这些长距离连接是通过光缆连接城市、国家和大陆的高速数据管道。这些电缆支持一种称为波分复用的网络技术，可以通过 400 个不同的通道传输高达 171 Gbps 的数据，为单个光纤链路提供每秒超过 70 太比特 (Tbps) 的总带宽。跨越世界的光缆通常包含四股或更多股光纤，每根光缆的带宽容量可达数百 Tbps。

WANs are networks that traverse the globe and make up what we collectively call the internet. These long-distance connections are the high speed data pipelines connecting cities, countries, and continents with fiber optic cables. These cables support a networking technology known as wavelength division multiplexing which makes it possible to transmit up to 171 Gbps over 400 different channels, giving more than 70 terabits per second (Tbps) of total bandwidth for a single fiber link. The fiber cables that span the world normally comprise four or more strands of fiber, giving bandwidth capacity of hundreds of Tbps for each cable.

延迟比较多然而，WAN 却很复杂。广域网传输数据的距离可达数百甚至数千公里，而数据在光纤电缆中传输的最大速度是理论光速。实际上，这些电缆无法达到光速，但确实非常接近光速，如表 3-1所示。

Latency is more complicated with WANs, however. WANs transmit data over hundreds and thousands of kilometers, and the maximum speed that the data can travel in fiber optic cables is the theoretical speed of light. In reality, these cables can’t reach the speed of light, but do get pretty close to it, as shown in Table 3-1.

表 3-1。广域网速度

小路	距离	旅行时间（光速）	行程时间（光缆）
纽约到旧金山	4,148 公里	14毫秒	21 毫秒
纽约到伦敦	5,585 公里	19 毫秒	28 毫秒
纽约飞往 悉尼	15,993 公里	53 毫秒	80毫秒

实际时间将比表 3-1中的光纤传输时间慢，因为数据需要通过称为路由器的网络设备。全球互联网具有复杂的中心辐射型拓扑结构，网络中的节点之间存在许多潜在路径。因此，路由器负责在物理网络连接上传输数据，以确保数据通过互联网从源传输到目的地。

Actual times will be slower than the fiber optic travel times in Table 3-1 as the data needs to pass through networking equipment known as routers. The global internet has a complex hub-and-spoke topology with many potential paths between nodes in the network. Routers are therefore responsible for transmitting data on the physical network connections to ensure data is transmitted across the internet from source to destination.

路由器是专业的、高速的设备可以处理数百 Gbps 的网络流量，从传入连接中提取数据，并根据目的地将数据发送到不同的传出网络连接。互联网核心的路由器由这些设备的机架组成，可以处理数十到数百 Tbps。这样您和成千上万的朋友就可以同时在 Netflix 上观看稳定的视频流。

Routers are specialized, high-speed devices that can handle several hundred Gbps of network traffic, pulling data off incoming connections and sending the data out to different outgoing network connections based on their destination. Routers at the core of the internet comprise racks of these devices and can process tens to hundreds of Tbps. This is how you and thousands of your friends get to watch a steady video stream on Netflix at the same time.

无线技术有不同的范围和带宽特性。我们在家庭和办公室中熟悉的 WiFi 路由器是无线以太网，使用 802.11 协议发送和接收数据。使用最广泛的 WiFi 协议 802.11ac 可实现高达 5,400 Mbps 的最大（理论）数据速率。最新的 802.11ax 协议（也称为 WiFi 6）是 802.11ac 技术的演进，有望将吞吐量速度提高至高达 9.6 Gbps。WiFi路由器的覆盖范围约为数十米，当然会受到墙壁和地板等物理障碍的影响。

Wireless technologies have different range and bandwidth characteristics. WiFi routers that we are all familiar with in our homes and offices are wireless Ethernet networks and use 802.11 protocols to send and receive data. The most widely used WiFi protocol, 802.11ac, allows for maximum (theoretical) data rates of up to 5,400 Mbps. The most recent 802.11ax protocol, also known as WiFi 6, is an evolution of 802.11ac technology that promises increased throughput speeds of up to 9.6 Gbps. The range of WiFi routers is of the order of tens of meters and of course is affected by physical impediments like walls and floors.

蜂窝无线技术的用途无线电波将数据从我们的手机发送到安装在手机信号塔上的路由器，这些路由器通常通过电线连接到核心互联网以进行消息路由。每种蜂窝技术都引入了改进的带宽和其他性能维度。在撰写本文时，最常见的技术是 4G LTE 无线宽带。4G LTE 的速度比旧 3G 快约 10 倍，能够处理约 10 Mbps 的持续下载速度（峰值下载速度接近 50 Mbps）和 2 至 5 Mbps 的上传速度。

Cellular wireless technology uses radio waves to send data from our phones to routers mounted on cell towers, which are generally connected by wires to the core internet for message routing. Each cellular technology introduces improved bandwidth and other dimensions of performance. The most common technology at the time of writing is 4G LTE wireless broadband. 4G LTE is around 10 times faster than the older 3G, able to handle sustained download speeds around 10 Mbps (peak download speeds are nearer 50 Mbps) and upload speeds between 2 and 5 Mbps.

新兴 5G 蜂窝网络承诺将现有 4G 的带宽提高 10 倍，设备和手机信号塔之间的延迟为 1-2 毫秒。与 20-40 毫秒范围内的 4G 延迟相比，这是一个巨大的改进。权衡是范围。5G 基站的工作范围最大约为 500 米，而 4G 则可在 10-15 公里的距离内提供可靠的接收。

Emerging 5G cellular networks promise 10x bandwidth improvements over existing 4G, with 1–2 millisecond latencies between devices and cell towers. This is a great improvement over 4G latencies, which are in the 20–40 millisecond range. The trade-off is range. 5G base station range operates at about 500 meters maximum, whereas 4G provides reliable reception at distances of 10–15 km.

用于网络的不同硬件类型的整个集合汇集在全球互联网中。互联网是一个异构网络，世界各地有许多不同的运营商以及可以想象的各种类型的硬件。图 3-1显示了构成互联网的主要组件的简化视图。一级网络是全球高速互联网骨干网。大约有 20 家一级互联网服务提供商 (ISP) 管理和控制全球流量。第 2 级 ISP 通常是区域性的（例如，一个国家/地区），其带宽低于第 1 级 ISP，并通过第 3 级 ISP 向客户提供内容。第三级 ISP 每月向您收取过高的家庭互联网费用。

This whole collection of different hardware types for networking comes together in the global internet. The internet is a heterogeneous network, with many different operators around the world and every type of hardware imaginable. Figure 3-1 shows a simplified view of the major components that comprise the internet. Tier 1 networks are the global high-speed internet backbone. There are around 20 Tier 1 internet service providers (ISPs) who manage and control global traffic. Tier 2 ISPs are typically regional (e.g., one country), have lower bandwidth than Tier 1 ISPs, and deliver content to customers through Tier 3 ISPs. Tier 3 ISPs are the ones that charge you exorbitant fees for your home internet every month.

图 3-1。互联网的简化视图

互联网的工作原理比这里描述的要复杂得多。网络和协议的复杂程度超出了本章的范围。从分布式系统软件的角度来看，我们需要更多地了解启用所有这些硬件的“魔力”例如，将消息从我的手机传送到我的银行并返回。这就是互联网协议（IP）的用武之地。

There’s a lot more complexity to how the internet works than described here. That level of networking and protocol complexity is beyond the scope of this chapter. From a distributed systems software perspective, we need to understand more about the “magic” that enables all this hardware to route messages from, say, my cell phone to my bank and back. This is where the Internet Protocol (IP) comes in.

通讯软件

Communications Software

互联网上的软件系统使用互联网协议（IP）套件进行通信。IP 套件指定主机寻址、数据传输格式、消息路由和传送特性。有四个抽象层，其中包含支持该层所需功能的相关协议。这些是，从最低到最高：

Software systems on the internet communicate using the Internet Protocol (IP) suite. The IP suite specifies host addressing, data transmission formats, message routing, and delivery characteristics. There are four abstract layers, which contain related protocols that support the functionality required at that layer. These are, from lowest to highest:

1. 数据链路层，指定通信跨单个网段的数据的方法。这是由设备内部的设备驱动程序和网卡实现的。

2. The data link layer, specifying communication methods for data across a single network segment. This is implemented by the device drivers and network cards that live inside your devices.

3. 互联网层指定寻址和路由协议，使流量能够穿越构成互联网的独立管理和控制的网络。这是互联网协议族中的IP 层。

4. The internet layer specifies addressing and routing protocols that make it possible for traffic to traverse the independently managed and controlled networks that comprise the internet. This is the IP layer in the internet protocol suite.

5. 传输层，指定用于可靠且尽力而为的主机到主机通信的协议。这就是著名的传输控制协议（TCP）和用户数据报协议（UDP）的所在地。

6. The transport layer, specifying protocols for reliable and best-effort, host-to-host communications. This is where the well-known Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) live.

7. 应用层，其中包括多个应用程序级协议，例如 HTTP 和安全复制协议 (SCP)。

8. The application layer, which comprises several application-level protocols such as HTTP and the secure copy protocol (SCP).

每个较高层协议都建立在较低层的功能之上。在下一节中，我将简要介绍用于主机发现和消息路由的 IP，以及分布式应用程序可以使用的 TCP 和 UDP。

Each of the higher-layer protocols builds on the features of the lower layers. In the following section, I’ll briefly cover IP for host discovery and message routing, and TCP and UDP that can be utilized by distributed applications.

互联网协议 (IP)

Internet Protocol (IP)

IP 定义主机如何分配地址互联网上的信息以及如何在知道彼此地址的两个主机之间传输消息。

IP defines how hosts are assigned addresses on the internet and how messages are transmitted between two hosts who know each other’s addresses.

互联网上的每个设备都有自己的地址。这些称为互联网协议 (IP) 地址。这可以使用称为域名系统 (DNS) 的互联网范围的目录服务找到 IP 地址的位置。DNS 是一种广泛分布的分层数据库，充当互联网的地址簿。

Every device on the internet has its own address. These are known as Internet Protocol (IP) addresses. The location of an IP address can be found using an internet-wide directory service known as Domain Name System (DNS). DNS is a widely distributed, hierarchical database that acts as the address book of the internet.

目前用于分配IP的技术地址，称为互联网协议版本 4 (IPv4)，最终将被其后继者 IPv6 所取代。IPv4 是一种 32 位寻址方案，由于连接到互联网的设备数量过多，不久之后就会耗尽地址。IPv6 是一种 128 位方案，将提供（几乎）无限数量的 IP 地址。作为一项指标，2020 年 7 月Google.com 处理的流量中约有 33%是 IPv6。

The technology currently used to assign IP addresses, known as Internet Protocol version 4 (IPv4), will eventually be replaced by its successor, IPv6. IPv4 is a 32-bit addressing scheme that before long will run out of addresses due to the number of devices connecting to the internet. IPv6 is a 128-bit scheme that will offer an (almost) infinite number of IP addresses. As an indicator, in July 2020 about 33% of the traffic processed by Google.com is IPv6.

DNS服务器是有组织的分层地。少量高度复制的根 DNS 服务器是解析 IP 地址的起点。当互联网浏览器尝试查找网站时，称为本地 DNS 服务器（由您的雇主或 ISP 管理）的网络主机将使用所请求的主机名联系根 DNS 服务器。根服务器回复一个所谓的权威DNS 服务器的引用，该服务器管理.com地址的名称解析（在我们的银行示例中）。每个顶级互联网域（.com、.org、.net等）都有一个权威名称服务器。

DNS servers are organized hierarchically. A small number of root DNS servers, which are highly replicated, are the starting point for resolving an IP address. When an internet browser tries to find a website, a network host known as the local DNS server (managed by your employer or ISP) will contact a root DNS server with the requested hostname. The root server replies with a referral to a so-called authoritative DNS server that manages name resolution for, in our banking example, .com addresses. There is an authoritative name server for each top-level internet domain (.com, .org, .net, etc.).

接下来，本地 DNS 服务器将查询.com DNS 服务器，后者将回复知道igbank.com管理的所有 IP 地址的 DNS 服务器的地址。查询此 DNS，它会返回我们与应用程序通信所需的实际 IP 地址。总体方案如图3-2所示。

Next, the local DNS server will query the .com DNS server, which will reply with the address of the DNS server that knows about all the IP addresses managed by igbank.com. This DNS is queried, and it returns the actual IP address we need to communicate with the application. The overall scheme is illustrated in Figure 3-2.

图 3-2。igbank.com 的 DNS 查找示例

整个DNS数据库高度地理上复制，因此不存在单点故障，并且请求分布在多个物理服务器上。本地 DNS 服务器还会记住最近联系的主机的 IP 地址，这是可能的，因为 IP 地址不会经常更改。这意味着我们联系的每个站点都不会发生完整的名称解析过程。

The whole DNS database is highly geographically replicated so there are no single points of failure, and requests are spread across multiple physical servers. Local DNS servers also remember the IP addresses of recently contacted hosts, which is possible as IP addresses don’t change very often. This means the complete name resolution process doesn’t occur for every site we contact.

有了目标 IP 地址，主机就可以开始通过网络以一系列 IP 数据包的形式发送数据。IP 根据数据包标头中的 IP 地址将数据从源主机传送到目标主机。IP 定义了一个数据包结构，其中包含要传送的数据以及包括源和目标 IP 地址的标头数据。应用程序发送的数据被分解为一系列数据包，这些数据包在互联网上独立传输。

Armed with a destination IP address, a host can start sending data across the network as a series of IP data packets. IP delivers data from the source to the destination host based on the IP addresses in the packet headers. IP defines a packet structure that contains the data to be delivered, along with header data including source and destination IP addresses. Data sent by an application is broken up into a series of packets which are independently transmitted across the internet.

IP 被称为尽力交付协议。这意味着它不会尝试补偿数据包传输期间可能发生的各种错误情况。可能的传输错误包括数据损坏、数据包丢失和重复。此外，每个数据包都通过互联网从源到目的地独立路由。独立处理每个数据包称为数据包交换。这使得网络能够动态响应网络链路故障和拥塞等情况，因此是互联网的一个定义特征。然而，这确实意味着不同的数据包可能通过不同的网络路径传递到同一目的地，从而导致无序传递到接收者。

IP is known as a best-effort delivery protocol. This means it does not attempt to compensate for the various error conditions that can occur during packet transmission. Possible transmission errors include data corruption, packet loss, and duplication. In addition, every packet is routed across the internet from source to destination independently. Treating every packet independently is known as packet switching. This allows the network to dynamically respond to conditions such as network link failure and congestion, and hence is a defining characteristic of the internet. This does mean, however, that different packets may be delivered to the same destination via different network paths, resulting in out-of-order delivery to the receiver.

由于这种设计，IP 不可靠。如果两台主机需要可靠的数据传输，则它们需要添加额外的功能来实现这一点。这是IP协议的下一层传输层套件登场。

Because of this design, the IP is unreliable. If two hosts require reliable data transmission, they need to add additional features to make this occur. This is where the next layer in the IP protocol suite, the transport layer, enters the scene.

传输控制协议 (TCP)

Transmission Control Protocol (TCP)

一旦应用程序或浏览器发现它希望与之通信的服务器的 IP 地址后，它可以使用传输协议 API 发送消息。这是使用 TCP 或 UDP 来实现的，它们是 IP 网络堆栈的流行标准传输协议。

Once an application or browser has discovered the IP address of the server it wishes to communicate with, it can send messages using a transport protocol API. This is achieved using TCP or UDP, which are the popular standard transport protocols for the IP network stack.

分布式应用程序可以选择使用这些协议中的哪一个。其实现广泛适用于 Java、Python 和 C++ 等主流编程语言。实际上，这些 API 的使用并不常见，因为高级编程抽象对大多数应用程序隐藏了细节。事实上，IP协议族应用层包含了几个这样的应用级API，其中包括在主流分布式系统中应用非常广泛的HTTP。

Distributed applications can choose which of these protocols to use. Implementations are widely available in mainstream programming languages such as Java, Python, and C++. In reality, use of these APIs is not common as higher-level programming abstractions hide the details from most applications. In fact, the IP protocol suite application layer contains several of these application-level APIs, including HTTP, which is very widely used in mainstream distributed systems.

尽管如此，了解 TCP、UDP 及其差异仍然很重要。Internet 上的大多数请求都是使用 TCP 发送的。TCP 是：

Still, it’s important to understand TCP, UDP, and their differences. Most requests on the internet are sent using TCP. TCP is:

• 面向连接

• Connection-oriented

• 面向流

• Stream-oriented

• 可靠的

• Reliable

我将在下面解释每一个品质以及它们为何重要。

I’ll explain each of these qualities, and why they matter, below.

TCP 被称为面向连接的协议。在应用程序之间交换任何消息之前，TCP 使用三步握手在客户端和服务器应用程序之间建立双向连接。连接保持打开状态，直到 TCP 客户端close()调用终止与 TCP 服务器的连接。服务器close()在连接断开之前通过确认请求进行响应。

TCP is known as a connection-oriented protocol. Before any messages are exchanged between applications, TCP uses a three-step handshake to establish a two-way connection between the client and server applications. The connection stays open until the TCP client calls close() to terminate the connection with the TCP server. The server responds by acknowledging the close() request before the connection is dropped.

一旦建立连接，客户端就会以数据流的形式向服务器发送一系列请求。当数据流通过 TCP 发送时，它会被分解为单独的网络数据包，最大数据包大小为 65,535 字节。每个数据包都包含一个源地址和目标地址，底层 IP 协议使用这些地址在网络上路由消息。

Once a connection is established, a client sends a sequence of requests to the server as a data stream. When a data stream is sent over TCP, it is broken up into individual network packets, with a maximum packet size of 65,535 bytes. Each packet contains a source and destination address, which is used by the underlying IP protocol to route the messages across the network.

互联网是一个数据包交换网络，这意味着每个数据包都在网络上单独路由。每个数据包经过的路由可以根据网络条件（例如链路拥塞或故障）动态变化。这意味着数据包可能不会按照从客户端发送的顺序到达服务器。为了解决这个问题，TCP 发送方在每个数据包中包含一个序列号，以便接收方可以将数据包重新组装成与发送顺序相同的流。

The internet is a packet switched network, which means every packet is individually routed across the network. The route each packet traverses can vary dynamically based on the conditions in the network, such as link congestion or failure. This means the packets may not arrive at the server in the same order they are sent from the client. To solve this problem, a TCP sender includes a sequence number in each packet so the receiver can reassemble packets into a stream that is identical to the order they were sent.

由于网络数据包在发送方和接收方之间传输期间可能会丢失或延迟，因此需要可靠性。为了实现可靠的数据包传送，TCP 使用累积确认机制。这意味着接收方将定期发送确认数据包，其中包含接收到的数据包的最高序列号，且数据包流中没有间隙。这隐式确认所有使用较低序列号发送的数据包，意味着所有数据包均已成功接收。如果发送方在超时时间内未收到确认，则会重新发送数据包。

Reliability is needed as network packets can be lost or delayed during transmission between sender and receiver. To achieve reliable packet delivery, TCP uses a cumulative acknowledgment mechanism. This means a receiver will periodically send an acknowledgment packet that contains the highest sequence number of the packets received without gaps in the packet stream. This implicitly acknowledges all packets sent with a lower sequence number, meaning all have been successfully received. If a sender doesn’t receive an acknowledgment within a timeout period, the packet is resent.

TCP 还有许多其他功能，例如用于检查数据包完整性的校验和，以及用于确保发送方不会因发送数据过快而压垮慢速接收方的动态流量控制。与连接建立和确认一起，这使得 TCP 成为相对重量级的协议，它牺牲了可靠性和效率。

TCP has many other features, such as checksums to check packet integrity, and dynamic flow control to ensure a sender doesn’t overwhelm a slow receiver by sending data too quickly. Along with connection establishment and acknowledgments, this makes TCP a relatively heavyweight protocol, which trades off reliability over efficiency.

这就是 UDP 发挥作用的地方。UDP 是一种简单的、无连接的协议，它使用户的程序暴露于底层网络的任何不可靠性。无法保证交付会按照规定的顺序进行，或者根本不会发生。它可以被认为是底层 IP 协议之上的一层薄单板（层），并故意在可靠性与原始性能之间进行权衡。

This is where UDP comes into the picture. UDP is a simple, connectionless protocol, which exposes the user’s program to any unreliability of the underlying network. There is no guarantee that delivery will occur in a prescribed order, or that it will happen at all. It can be thought of as a thin veneer (layer) on top of the underlying IP protocol, and deliberately trades off raw performance over reliability.

然而，这非常适合许多现代应用程序，其中奇数丢失的数据包影响很小。想想流媒体电影、视频会议和游戏，用户不太可能察觉到丢失的数据包。

This, however, is highly appropriate for many modern applications where the odd lost packet has very little effect. Think streaming movies, video conferencing, and gaming, where one lost packet is unlikely to be perceptible by a user.

图 3-3描述了 TCP 和 UDP 之间的一些主要区别。TCP 包含连接建立三数据包握手（SYN、SYN ACK、ACK）和数据包捎带确认（ACK），以便协议可以处理任何数据包丢失。还有一个 TCP 连接关闭阶段，涉及四次握手（图中未显示）。UDP 无需连接建立、断开、确认和重试。因此，使用UDP的应用程序需要容忍丢包，客户端或服务器失败（并采取相应的行为）。

Figure 3-3 depicts some of the major differences between TCP and UDP. TCP incorporates a connection establishment three-packet handshake (SYN, SYN ACK, ACK), and piggybacks acknowledgments (ACK) of packets so that any packet loss can be handled by the protocol. There’s also a TCP connection close phase involving a four-way handshake that is not shown in the diagram. UDP dispenses with connection establishment, tear down, acknowledgments, and retries. Therefore, applications using UDP need to be tolerant of packet loss and client or server failures (and behave accordingly).

图 3-3。比较 TCP 和 UDP

远程方法调用

Remote Method Invocation

使用直接交互的低级 API 来编写分布式应用程序是完全可行的传输层协议 TCP 和 UDP。最常见的方法是标准化套接字库 - 请参阅侧栏中的简要概述。这是您希望永远不需要做的事情，因为套接字很复杂并且容易出错。本质上，套接字在两个节点之间创建一个双向管道，可用于发送数据流。（幸运的是）有更好的方法来构建分布式通信，正如我将在本节中描述的那样。这些方法消除了使用套接字的大部分复杂性。然而，套接字仍然潜伏在下面，所以一些知识是必要的。

It’s perfectly feasible to write our distributed applications using low-level APIs that interact directly with the transport layer protocols TCP and UDP. The most common approach is the standardized sockets library—see the brief overview in the sidebar. This is something you’ll hopefully never need to do, as sockets are complex and error prone. Essentially, sockets create a bidirectional pipe between two nodes that you can use to send streams of data. There are (luckily) much better ways to build distributed communications, as I’ll describe in this section. These approaches abstract away much of the complexity of using sockets. However, sockets still lurk underneath, so some knowledge is necessary.

在我们的移动银行示例中，客户端可能会使用套接字请求用户支票帐户的余额。忽略特定的语言问题（和安全性！），客户端可以通过与服务器的连接发送如下消息有效负载：

In our mobile banking example, the client might request a balance for the user’s checking account using sockets. Ignoring specific language issues (and security!), the client could send a message payload as follows over a connection to the server:

{“余额”，“000169990”}

{“balance”, “000169990”}

在这条消息中，“balance”代表我们希望服务器执行的操作，“000169990”是银行帐号。

In this message, “balance” represents the operation we want the server to execute, and “000169990” is the bank account number.

在服务器中，我们需要知道消息中的第一个字符串是操作标识符，基于该值为“余额”，第二个字符串是银行帐号。然后，服务器可能会使用这些值来查询数据库、检索余额并发回结果，可能会以包含帐号和当前余额的消息的形式发回，如下所示：

In the server, we need to know that the first string in the message is the operation identifier, and based on this value being “balance”, the second is the bank account number. The server then uses these values to presumably query a database, retrieve the balance, and send back the results, perhaps as a message formatted with the account number and current balance, as below:

{“000169990”,“220.77”}

{“000169990”, “220.77”}

在任何复杂的系统中，服务器都会支持许多操作。在igbank.com中，可能有“登录”、“转账”、“地址”、“声明”、“交易”等。每个消息后面都会跟着不同的消息有效负载，服务器需要正确解释这些消息负载才能满足客户端的请求。

In any complex system, the server will support many operations. In igbank.com, there might be for example “login”, “transfer”, “address”, “statement”, “transactions”, and so on. Each will be followed by different message payloads that the server needs to interpret correctly to fulfill the client’s request.

我们在这里定义的是特定于应用程序的协议。只要我们为每个操作以正确的顺序发送必要的值，服务器就能够正确响应。如果我们有一个不遵守我们的应用程序协议的错误客户端，那么我们的服务器需要进行彻底的错误检查。套接字库为客户端/服务器通信提供了一种原始的低级方法。它提供了高效通信，但很难正确实现和发展应用程序协议来处理所有可能性。有更好的机制。

What we are defining here is an application-specific protocol. As long as we send the necessary values in the correct order for each operation, the server will be able to respond correctly. If we have an erroneous client that doesn’t adhere to our application protocol, well, our server needs to do thorough error checking. The socket library provides a primitive, low-level method for client/server communications. It provides highly efficient communications but is difficult to correctly implement and evolve the application protocol to handle all possibilities. There are better mechanisms.

退一步来说，如果我们用面向对象的语言（例如 Java）定义igbank.com服务器接口，我们会将它可以处理的每个操作作为一个方法。每个方法都会传递一个适合该操作的参数列表，如以下示例代码所示：

Stepping back, if we were defining the igbank.com server interface in an object-oriented language such as Java, we would have each operation it can process as a method. Each method is passed an appropriate parameter list for that operation, as shown in this example code:

// 简单的 igbank.com 服务器接口
公共接口 IGBank {
    公共浮动余额（String accNo）；
    公共布尔语句（字符串月）；
    // 其他操作
}

// Simple igbank.com server interface
public interface IGBank {
    public float balance  (String accNo);
    public boolean  statement(String month) ;
    // other operations
}

拥有这样的接口有几个优点，即：

There are several advantages of having such an interface, namely:

• 编译器可以静态检查从客户端到服务器的调用，以确保它们具有正确的格式和参数类型。

• Calls from the client to the server can be statically checked by the compiler to ensure they are of the correct format and argument types.

• 服务器接口的更改（例如，添加新参数）强制客户端代码中的更改遵循新的方法签名。

• Changes in the server interface (e.g., adding a new parameter) force changes in the client code to adhere to the new method signature.

• 该接口由类定义明确定义，因此客户端程序员可以轻松理解和使用。

• The interface is clearly defined by the class definition and thus straightforward for a client programmer to understand and utilize.

显式接口的这些好处在软件工程中当然是众所周知的。面向对象设计的整个规则很大程度上基于这些基础，其中接口定义了调用者和被调用者之间的契约。与我们使用套接字需要遵循的隐式应用程序协议相比，优点是显着的。

These benefits of an explicit interface are of course well known in software engineering. The whole discipline of object-oriented design is pretty much based upon these foundations, where an interface defines a contract between the caller and callee. Compared to the implicit application protocol we need to follow with sockets, the advantages are significant.

这一事实在分布式系统创建之初就得到了合理的认识。自 20 世纪 90 年代初以来，我们看到了技术的发展，这些技术使我们能够定义显式的服务器接口，并使用与顺序程序中基本相同的语法在网络上调用这些接口。表 3-2总结了主要方法。总的来说，他们是众所周知的例如远程过程调用 (RPC) 或远程方法调用 (RMI) 技术。

This fact was recognized reasonably early in the creation of distributed systems. Since the early 1990s, we have seen an evolution of technologies that enable us to define explicit server interfaces and call these across the network using essentially the same syntax as we would in a sequential program. A summary of the major approaches is given in Table 3-2. Collectively, they are known as Remote Procedure Call (RPC), or Remote Method Invocation (RMI) technologies.

虽然这些 RPC/RMI 技术的语法和语义各不相同，但每种技术的操作本质是相同的。让我们继续使用igbank.com的 Java 示例来检查整个方法类。Java 提供了用于构建客户端/服务器应用程序的远程方法调用 (RMI) API。

While the syntax and semantics of these RPC/RMI technologies vary, the essence of how each operates is the same. Let’s continue with our Java example of igbank.com to examine the whole class of approaches. Java offers a Remote Method Invocation (RMI) API for building client/server applications.

使用 Java RMI，我们可以轻松地将IGBank上面的接口示例变成远程接口，如以下代码所示：

Using Java RMI, we can trivially make our IGBank interface example from above into a remote interface, as illustrated in the following code:

导入java.rmi.*；
// 简单的 igbank.com 服务器接口
公共接口 IGBank extends Remote{
    公共浮动余额（字符串 accNo）
         投掷RemoteException;
    公共布尔语句（字符串月）
         投掷RemoteException ;
    // 其他操作
 }

import java.rmi.*;
// Simple igbank.com server interface
public interface IGBank extends Remote{
    public float balance  (String accNo)
         throws RemoteException;
    public boolean  statement(String month)
         throws RemoteException ;
    // other operations
 }

该java.rmi.Remote接口充当一个标记，通知 Java 编译器我们正在创建一个 RMI 服务器。此外，每个方法都必须抛出java.rmi.RemoteException。这些异常表示通过网络调用两个对象之间的分布式调用时可能发生的错误。此类异常的最常见原因是通信故障或服务器对象崩溃。

The java.rmi.Remote interface serves as a marker to inform the Java compiler we are creating an RMI server. In addition, each method must throw java.rmi.RemoteException. These exceptions represent errors that can occur when a distributed call between two objects is invoked over a network. The most common reasons for such an exception would be a communications failure or the server object having crashed.

然后我们必须提供一个实现此远程接口的类。下面的示例代码显示了服务器实现的摘录：

We then must provide a class that implements this remote interface. The sample code below shows an extract of the server implementation:

公共类 IGBankServer 扩展 UnicastRemoteObject
                          实施 IGBank {
   // 省略构造函数/方法实现
   公共静态无效主（字符串参数[]）{  
        尝试{  
          IGBankServer 服务器 = new IGBankServer();  
          // 在默认端口上的本地 JVM 中创建注册表
          注册表registry = LocateRegistry.createRegistry(1099);
          registry.bind("IGBankServer", 服务器);
          System.out.println("服务器就绪");
        }捕获（异常e）{
                 // 为简洁起见省略代码}  
        }  
   }

public class IGBankServer extends UnicastRemoteObject 
                          implements IGBank  {
   // constructor/method implementations omitted
   public static void main(String args[]){  
        try{  
          IGBankServer server=new IGBankServer();  
          // create a registry in local JVM on default port
          Registry registry = LocateRegistry.createRegistry(1099);
          registry.bind("IGBankServer", server);
          System.out.println("server ready");
        }catch(Exception e){
                 // code omitted for brevity}  
        }  
   }

需要注意的点是：

Points to note are:

• 服务器扩展了UnicastRemoteObject类。这本质上提供了实例化远程可调用对象的功能。

• The server extends the UnicastRemoteObject class. This essentially provides the functionality to instantiate a remotely callable object.

• 一旦构建了服务器对象，就必须将其可用性通告给远程客户端。这是通过在系统中存储对该对象的引用来实现的称为RMI 注册表的服务，并将逻辑名称与其关联起来 — 在本例中为“ IGBankServer ”。注册表是一种简单的目录服务，使客户端能够通过简单地提供与注册表中关联的逻辑名称来查找 RMI 服务器的位置（网络地址和对象引用）并获取对 RMI 服务器的引用。

• Once the server object is constructed, its availability must be advertised to remote clients. This is achieved by storing a reference to the object in a system service known as the RMI registry, and associating a logical name with it—in this example, “IGBankServer.” The registry is a simple directory service that enables clients to look up the location (network address and object reference) of and obtain a reference to an RMI server by simply supplying the logical name it is associated with in the registry.

以下示例显示了连接到服务器的客户端代码的摘录。lookup它通过在 RMI 注册表中执行操作并指定标识服务器的逻辑名称来获取对远程对象的引用。然后，可以使用查找操作返回的引用以与本地对象相同的方式调用服务器对象。然而，有一个区别——客户端必须准备好捕获RemoteException当无法到达服务器对象时 Java 运行时将抛出的异常：

An extract from the client code to connect to the server is shown in the following example. It obtains a reference to the remote object by performing a lookup operation in the RMI registry and specifying the logical name that identifies the server. The reference returned by the lookup operation can then be used to call the server object in the same manner as a local object. However, there is a difference—the client must be ready to catch a RemoteException that will be thrown by the Java runtime when the server object cannot be reached:

// 获取对服务器的远程引用
 IGBank银行服务器=
        (IGBank)Naming.lookup("rmi://localhost:1099/IGBankServer");  
 //现在我们可以调用服务器了
 System.out.println(bankServer.balance("00169990"));

 // obtain a remote reference to the server
 IGBank bankServer=
        (IGBank)Naming.lookup("rmi://localhost:1099/IGBankServer");  
 //now we can call the server
 System.out.println(bankServer.balance("00169990"));

图 3-4描述了组成 RMI 系统的组件之间的调用顺序。和是编译器根据 RMI 接口定义生成的对象，这些对象有助于实际的远程通信Stub。Skeleton该框架实际上是一个 TCP 网络端点（主机、端口），用于侦听对关联服务器的调用。

Figure 3-4 depicts the call sequence among the components that comprise an RMI system. The Stub and Skeleton are objects generated by the compiler from the RMI interface definition, and these facilitate the actual remote communications. The skeleton is in fact a TCP network endpoint (host, port) that listens for calls to the associated server.

图 3-4。描述建立连接和调用 RMI 服务器对象的调用序列的示意图

操作顺序如下：

The sequence of operations is as follows:

1. 当服务器启动时，其逻辑引用存储在 RMI 注册表中。该条目包含可用于对服务器进行远程调用的 Java 客户端存根。

2. When the server starts, its logical reference is stored in the RMI registry. This entry contains the Java client stub that can be used to make remote calls to the server.

3. 客户端查询注册表，并返回服务器的存根。

4. The client queries the registry, and the stub for the server is returned.

5. 客户端存根接受来自 Java 客户端实现的对服务器接口的方法调用。

6. The client stub accepts a method call to the server interface from the Java client implementation.

7. 存根将请求转换为一个或多个发送到服务器主机的网络数据包。这个转换过程称为编组。

8. The stub transforms the request into one or more network packets that are sent to the server host. This transformation process is known as marshalling.

9. 该框架接受来自客户端的网络请求，并将网络数据包数据解组为对 RMI 服务器对象实现的有效调用。解组与编组相反，它采用一系列网络数据包并将它们转换为对对象的调用。

10. The skeleton accepts network requests from the client, and unmarshalls the network packet data into a valid call to the RMI server object implementation. Unmarshalling is the opposite of marshalling—it takes a sequence of network packets and transforms them into a call to an object.

11. 骨架等待方法返回响应。

12. The skeleton waits for the method to return a response.

13. 骨架将方法结果编组到返回给客户端的网络回复数据包中。

14. The skeleton marshalls the method results into a network reply packet that is returned to the client.

15. 存根解组数据并将结果传递到 Java 客户端调用站点。

16. The stub unmarshalls the data and passes the result to the Java client call site.

这个 Java RMI 示例说明了用于实现任何 RPC/RMI 机制的基础知识，甚至在Erlang和Go等现代语言中也是如此。在使用 Java Enterprise JavaBeans (EJB) 技术时，您最有可能遇到 Java RMI。EJB 是一个基于 RMI 构建的服务器端组件模型，在过去 20 年左右的时间里在企业系统中得到了广泛的应用。

This Java RMI example illustrates the basics that are used for implementing any RPC/RMI mechanism, even in modern languages like Erlang and Go. You are most likely to encounter Java RMI when using the Java Enterprise JavaBeans (EJB) technology. EJBs are a server-side component model built on RMI, which have seen wide usage in the last 20 or so years in enterprise systems.

不管具体的实现如何，RPC/RMI 方法的基本吸引力在于提供一种抽象的调用机制，支持客户端进行远程服务器调用的位置透明性。位置透明性由注册表提供，或者通常由使客户端能够通过目录服务定位服务器的任何机制提供。这意味着服务器可以更新其在目录中的网络位置，而不会影响客户端实现。

Regardless of the precise implementation, the basic attraction of RPC/RMI approaches is to provide an abstract calling mechanism that supports location transparency for clients making remote server calls. Location transparency is provided by the registry, or in general any mechanism that enables a client to locate a server through a directory service. This means it is possible for the server to update its network location in the directory without affecting the client implementation.

RPC/RMI 并非没有缺陷。对于复杂的对象参数，编组和解编组可能会变得低效。跨语言编组（客户端使用一种语言，服务器使用另一种语言）可能会导致问题，因为类型在不同语言中的表示方式不同，从而导致微妙的不兼容性。如果远程方法签名发生更改，所有客户端都需要获取新的兼容存根，这在大型部署中可能会很麻烦。

RPC/RMI is not without its flaws. Marshalling and unmarshalling can become inefficient for complex object parameters. Cross-language marshalling—client in one language, server in another—can cause problems due to types being represented differently in different languages, causing subtle incompatibilities. And if a remote method signature changes, all clients need to obtain a new compatible stub, which can be cumbersome in large deployments.

由于这些原因，大多数现代系统都是围绕基于 HTTP 的更简单的协议构建的，并使用 JSON 进行参数表示。HTTP 动词（PUT、GET、POST等）具有映射到特定 URL 的关联语义，而不是操作名称。这种方法起源于 Roy Fielding 关于 REST 方法的工作。¹ REST 具有一组语义，构成RESTful架构风格，并且实际上大多数系统不遵守这些。我们将在第 5 章中讨论 REST 和 HTTP API 机制。

For these reasons, most modern systems are built around simpler protocols based on HTTP and using JSON for parameter representation. Instead of operation names, HTTP verbs (PUT, GET, POST, etc.) have associated semantics that are mapped to a specific URL. This approach originated in the work by Roy Fielding on the REST approach.¹ REST has a set of semantics that comprise a RESTful architecture style, and in reality most systems do not adhere to these. We’ll discuss REST and HTTP API mechanisms in Chapter 5.

部分故障

Partial Failures

分布式系统的组件通过网络进行通信。在通信技术术语中，我们的系统通过共享的局域网和广域网进行通信，称为异步网络。

The components of distributed systems communicate over a network. In communications technology terminology, the shared local and wide area networks that our systems communicate over are known as asynchronous networks.

对于异步网络：

With asynchronous networks:

• 节点可以随时选择向其他节点发送数据。

• Nodes can choose to send data to other nodes at any time.

• 网络是半双工的，这意味着一个节点发送请求并且必须等待另一节点的响应。这是两个单独的通信。

• The network is half-duplex, meaning that one node sends a request and must wait for a response from the other. These are two separate communications.

• 由于网络拥塞、动态数据包路由和瞬时网络连接故障等原因，节点之间传输数据的时间是可变的。

• The time for data to be communicated between nodes is variable, due to reasons like network congestion, dynamic packet routing, and transient network connection failures.

• 由于软件或机器崩溃，接收节点可能不可用。

• The receiving node may not be available due to a software or machine crash.

• 数据可能会丢失。在无线网络中，数据包可能会因信号较弱或干扰而损坏并因此被丢弃。互联网路由器可能会在拥塞期间丢弃数据包。

• Data can be lost. In wireless networks, packets can be corrupted and hence dropped due to weak signals or interference. Internet routers can drop packets during congestion.

• 节点没有相同的内部时钟；因此它们不同步。

• Nodes do not have identical internal clocks; hence they are not synchronized.

这对我们的应用程序意味着什么？那么，简单来说，当客户端向服务器发送请求时，要等待多长时间才能收到回复？只是服务器节点速度慢吗？网络是否拥塞并且数据包已被路由器丢弃？如果客户没有得到回复，该怎么办？

What does this mean for our applications? Well, put simply, when a client sends a request to a server, how long does it wait until it receives a reply? Is the server node just being slow? Is the network congested and the packet has been dropped by a router? If the client doesn’t get a reply, what should it do?

让我们详细探讨这些场景。这里的核心问题，即是否以及何时收到响应，被称为处理部分失败，一般情况如图3-5所示。

Let’s explore these scenarios in detail. The core problem here, namely whether and when a response is received, is known as handling partial failures, and the general situation is depicted in Figure 3-5.

图 3-5。处理部分失败

当客户端希望连接到服务器并交换消息时，可能会发生以下结果：

When a client wishes to connect to a server and exchanges messages, the following outcomes may occur:

• 请求成功并收到快速响应。一切都很好。（实际上，几乎每个请求都会出现这种结果。几乎是这里的关键词。）

• The request succeeds and a rapid response is received. All is well. (In reality, this outcome occurs for almost every request. Almost is the operative word here.)

• 目标 IP 地址查找可能会失败。在这种情况下，客户端会快速收到错误消息并采取相应措施。

• The destination IP address lookup may fail. In this case, the client rapidly receives an error message and can act accordingly.

• IP 地址有效，但目标节点或目标服务器进程失败。发送者将收到超时错误消息并可以通知用户。

• The IP address is valid but the destination node or target server process has failed. The sender will receive a timeout error message and can inform the user.

• 目标服务器接收请求，但在处理请求时失败，并且从未发送任何响应。

• The request is received by the target server, which fails while processing the request and no response is ever sent.

• 该请求被负载较重的目标服务器接收。它处理请求但需要很长时间（例如，34 秒）来响应。

• The request is received by the target server, which is heavily loaded. It processes the request but takes a long time (e.g., 34 seconds) to respond.

• 目标服务器接收请求并发送响应。但由于网络故障，客户端收不到响应。

• The request is received by the target server and a response is sent. However, the response is not received by the client due to a network failure.

前三点对于客户来说很容易处理，因为可以很快收到响应。来自服务器的结果或错误消息——要么允许客户端继续。能够快速检测到的故障很容易处理。

The first three points are easy for the client to handle, as a response is received rapidly. A result from the server or an error message—either allows the client to proceed. Failures that can be detected quickly are easy to deal with.

其余的结果给客户带来了问题。他们没有提供任何有关未收到答复的原因的见解。从客户的角度来看，这三个结果看起来完全相同。如果不等待（可能永远），客户端就无法知道响应是否最终到达或永​​远不会到达；永远的等待并不能完成很多工作。

The rest of the outcomes pose a problem for the client. They do not provide any insight into the reason why a response has not been received. From the client’s perspective, these three outcomes look exactly the same. The client cannot know without waiting (potentially forever) whether the response will arrive eventually or never arrive; waiting forever doesn’t get much work done.

更阴险的是，客户端无法知道操作是否成功并且服务器或网络故障导致结果永远无法到达，或者请求是否正在路上（仅由于网络/服务器拥塞而延迟）。这些故障统称为崩溃故障。

More insidiously, the client cannot know if the operation succeeded and a server or network failure caused the result to never arrive, or if the request is on its way—delayed simply due to congestion in the network/server. These faults are collectively known as crash faults.

客户端处理崩溃故障的典型解决方案是在配置的超时时间后重新发送请求。然而，如图 3-6所示，这充满了危险。客户端向服务器发送请求，将钱存入银行帐户。当超时后未收到响应时，它会重新发送请求。由此产生的余额是多少？服务器可能已应用押金，也可能未应用押金，具体取决于部分故障情况。

The typical solution that clients adopt to handle crash faults is to resend the request after a configured timeout period. However, this is fraught with danger, as Figure 3-6 illustrates. The client sends a request to the server to deposit money in a bank account. When it receives no response after a timeout period, it resends the request. What is the resulting balance? The server may have applied the deposit, or it may not, depending on the partial failure scenario.

图 3-6。客户端超时后重试请求

存款可能发生两次对于客户来说是一个很好的结果，但银行不太可能对这种可能性感到好笑。因此，我们需要一种方法来确保在我们的服务器操作实现中重试来自客户端的重复请求只会导致该请求被应用一次。这对于维护正确的应用程序语义是必要的。

The chance that the deposit may occur twice is a fine outcome for the customer, but the bank is unlikely to be amused by this possibility. Therefore, we need a way to ensure in our server operations implementation that retried, duplicate requests from clients only result in the request being applied once. This is necessary to maintain correct application semantics.

此属性称为幂等性。幂等操作可以应用多次，而不会改变初始应用之外的结果。这意味着对于图 3-6中的示例，客户端可以多次重试请求，并且帐户只会增加 100 美元。

This property is known as idempotence. Idempotent operations can be applied multiple times without changing the result beyond the initial application. This means that for the example in Figure 3-6, the client can retry the request as many times as it likes, and the account will only be increased by $100.

不进行持久状态更改的请求自然是幂等的。这意味着所有读取请求本质上都是安全的，不需要在服务器上进行额外的工作。更新是另一回事。系统需要设计一种机制，使得重复的客户端请求不会导致任何状态变化，并且可以被服务器检测到。用 API 术语来说，这些端点会导致服务器状态发生变化，因此必须是幂等的。

Requests that make no persistent state changes are naturally idempotent. This means all read requests are inherently safe and no extra work is needed on the server. Updates are a different matter. The system needs to devise a mechanism such that duplicate client requests do not cause any state changes and can be detected by the server. In API terms, these endpoints cause mutation of the server state and must therefore be idempotent.

构建幂等操作的一般方法如下：

The general approach to building idempotent operations is as follows:

• 客户包括所有改变状态的请求中唯一的幂等键。该键标识来自特定客户端或事件源的单个操作。它通常是用户标识符（例如会话密钥）和唯一值（例如本地时间戳、UUID 或序列号）的组合。

• Clients include a unique idempotency key in all requests that mutate state. The key identifies a single operation from the specific client or event source. It is usually a composite of a user identifier, such as the session key, and a unique value such as a local timestamp, UUID, or a sequence number.

• 当服务器收到请求时，它会通过从专门为实现幂等性而设计的数据库中读取内容来检查之前是否已看到幂等性键值。如果密钥不在数据库中，则这是一个新请求。因此，服务器执行业务逻辑来更新应用程序状态。它还将幂等性密钥存储在数据库中以指示操作已成功应用。

• When the server receives a request, it checks to see if it has previously seen the idempotency key value by reading from a database that is uniquely designed for implementing idempotence. If the key is not in the database, this is a new request. The server therefore performs the business logic to update the application state. It also stores the idempotency key in a database to indicate that the operation has been successfully applied.

• 如果幂等性密钥在数据库中，则表明该请求是来自客户端的重试，因此不应被处理。在这种情况下，服务器返回操作的有效响应，以便（希望）客户端不会再次重试。

• If the idempotency key is in the database, this indicates that this request is a retry from the client and hence should not be processed. In this case the server returns a valid response for the operation so that (hopefully) the client won’t retry again.

用于存储幂等性密钥的数据库可以在例如：

The database used to store idempotency keys can be implemented in, for example:

• 事务数据库中用于应用程序数据的单独数据库表或集合

• A separate database table or collection in the transactional database used for the application data

• 提供极低延迟查找的专用数据库，例如简单的键值存储

• A dedicated database that provides very low latency lookups, such as a simple key-value store

与应用程序数据不同，幂等密钥不必永远保留。一旦客户端收到单个操作成功的确认，就可以丢弃幂等密钥。实现此目的的最简单方法是在特定时间段（例如 60 分钟或 24 小时）后自动从存储中删除幂等密钥，具体取决于应用程序需求和请求量。

Unlike application data, idempotency keys don’t have to be retained forever. Once a client receives an acknowledgment of a success for an individual operation, the idempotency key can be discarded. The simplest way to achieve this is to automatically remove idempotency keys from the store after a specific time period, such as 60 minutes or 24 hours, depending on application needs and request volumes.

此外，幂等 API 实现必须确保应用程序状态被修改并存储幂等密钥。两者都必须同时发生才能成功。如果应用程序状态被修改，并且由于某些故障而未存储幂等密钥，则重试将导致该操作被应用两次。如果存储了幂等性密钥但由于某种原因应用程序状态没有被修改，则该操作尚未应用。如果重试到达，则会将其作为重复项过滤掉，因为幂等性键已存在，并且更新将丢失。

In addition, an idempotent API implementation must ensure that the application state is modified and the idempotency key is stored. Both must occur for success. If the application state is modified and, due to some failure, the idempotent key is not stored, then a retry will cause the operation to be applied twice. If the idempotency key is stored but for some reason the application state is not modified, then the operation has not been applied. If a retry arrives, it will be filtered out as duplicate as the idempotency key already exists, and the update will be lost.

这里的含义是应用程序状态和幂等性密钥存储的更新必须同时发生，或者都不发生。如果您了解您的数据库，您就会认识到这是事务语义的要求。我们将在第 12 章讨论分布式事务是如何实现的。本质上，事务确保了操作的一次性语义，这保证了所有消息将始终被精确处理一次——精确地我们需要幂等性。

The implication here is that the updates to the application state and idempotency key store must both occur, or neither must occur. If you know your databases, you’ll recognize this as a requirement for transactional semantics. We’ll discuss how distributed transactions are achieved in Chapter 12. Essentially, transactions ensure exactly-once semantics for operations, which guarantees that all messages will always be processed exactly once—precisely what we need for idempotence.

恰好一次并不意味着没有消息传输失败、重试和应用程序崩溃。这些都是不可避免的。重要的是重试最终会成功，并且结果总是相同的。

Exactly once does not mean that there are no message transmission failures, retries, and application crashes. These are all inevitable. The important thing is that the retries eventually succeed and the result is always the same.

我们将在后面的章节中回到通信交付保证的问题。如图3-7所示，存在一系列语义，每种语义都有不同的保证和性能特征。最多一次传送是快速且不可靠的——这就是 UDP 协议所提供的。至少一次传送是 TCP/IP 提供的保证，这意味着重复是不可避免的。正如我们在这里讨论的，一次性交付需要防止重复，因此会牺牲可靠性和性能下降。

We’ll return to the issue of communications delivery guarantees in later chapters. As Figure 3-7 illustrates, there’s a spectrum of semantics, each with different guarantees and performance characteristics. At-most-once delivery is fast and unreliable—this is what the UDP protocol provides. At-least-once delivery is the guarantee provided by TCP/IP, meaning duplicates are inevitable. Exactly-once delivery, as we’ve discussed here, requires guarding against duplicates and hence trades off reliability against slower performance.

图 3-7。通讯传输保证

正如我们将看到的，一些高级通信机制可以为我们的应用程序提供一次性语义。然而，由于性能影响，这些不能在互联网规模上运行。这就是为什么，由于我们的应用程序是基于 TCP/IP 的至少一次语义构建的，因此我们必须在 API 中实现恰好一次语义，导致状态突变。

As we’ll see, some advanced communications mechanisms can provide our applications with exactly-once semantics. However, these don’t operate at internet scale because of the performance implications. That is why, as our applications are built on the at-least-once semantics of TCP/IP, we must implement exactly-once semantics in our APIs that cause state mutation.

分布式系统中的共识

Consensus in Distributed Systems

碰撞故障还有另一个含义我们构建分布式系统的方式。图 3-8中描绘的二将军问题最好地说明了这一点。

Crash faults have another implication for the way we build distributed systems. This is best illustrated by the Two Generals’ Problem, which is depicted in Figure 3-8.

图 3-8。两位将军的问题

想象一下一座城市被两支军队围困。军队分布在城市的两侧，城市周围的地形很难穿过，而且城内的狙击手也能看到。为了攻克这座城市，两军同时进攻至关重要。这将增强城市的防御力，使攻击者更有可能取得胜利。如果只有一支军队进攻，那么他们很可能会被击退。

Imagine a city under siege by two armies. The armies lie on opposite sides of the city, and the terrain surrounding the city is difficult to travel through and visible to snipers in the city. In order to overwhelm the city, it’s crucial that both armies attack at the same time. This will stretch the city’s defenses and make victory more likely for the attackers. If only one army attacks, then they will likely be repelled.

考虑到这些限制，两位将军如何才能就确切的进攻时间达成一致，从而让两位将军确信已经达成协议？他们都需要确定对方军队会在约定的时间发起进攻，否则灾难就会随之而来。

Given these constraints, how can the two generals reach agreement on the exact time to attack, such that both generals know for certain that agreement has been reached? They both need certainty that the other army will attack at the agreed time, or disaster will ensue.

为了协调攻击，第一位将军向另一位将军派遣一名信使，指示在特定时间进攻。由于信使可能会被狙击手俘获或杀死，因此发送的将军无法确定消息已到达，除非他们得到第二位将军的确认信使。当然，确认的使者可能会被俘虏或被杀，所以即使原来的使者确实成功了，第一将也可能永远不会知道。即使确认消息到达，第二个将军如何知道这一点，除非他们得到第一个将军的确认？

To coordinate an attack, the first general sends a messenger to the other, with instructions to attack at a specific time. As the messenger may be captured or killed by snipers, the sending general cannot be certain the message has arrived unless they get an acknowledgment messenger from the second general. Of course, the acknowledgment messenger may be captured or killed, so even if the original messenger does get through, the first general may never know. And even if the acknowledgment message arrives, how does the second general know this, unless they get an acknowledgment from the first general?

希望问题是显而易见的。由于信使被随机捕获或消灭，无法保证两位将军能够就进攻时间达成一致。事实上，可以证明，无法保证达成协议。有一些解决方案可以增加达成共识的可能性。例如，《权力的游戏》风格中，每个将军每次可能会派出100个不同的信使，即使大多数人被杀，这也会增加至少一个信使冒险前往另一支友军并成功传递消息的概率。

Hopefully the problem is apparent. With messengers being randomly captured or extinguished, there is no guarantee the two generals will ever reach consensus on the attack time. In fact, it can be proven that it is not possible to guarantee agreement will be reached. There are solutions that increase the likelihood of reaching consensus. For example, Game of Thrones style, each general may send 100 different messengers every time, and even if most are killed, this increases the probability that at least one will make the perilous journey to the other friendly army and successfully deliver the message.

双将军问题类似于分布式系统中的两个节点希望就某些状态达成一致，例如可以在任一位置更新的数据项的值。部分失败类似于丢失消息和确认。消息可能会丢失或延迟一段不确定的时间——这是异步网络的特征，正如我在本章前面所描述的那样。

The Two Generals’ Problem is analogous to two nodes in a distributed system wishing to reach agreement on some state, such as the value of a data item that can be updated at either. Partial failures are analogous to losing messages and acknowledgments. Messages may be lost or delayed for an indeterminate period of time—the characteristics of asynchronous networks, as I described earlier in this chapter.

事实上，可以证明，在存在崩溃故障的情况下，消息可以延迟但不会丢失的异步网络上的共识是不可能在有限的时间内达成的。这称为 FLP 不可能定理。²

In fact it can be demonstrated that consensus on an asynchronous network in the presence of crash faults, where messages can be delayed but not lost, is impossible to achieve within bounded time. This is known as the FLP Impossibility Theorem.²

幸运的是，这只是理论上的限制，表明不可能保证在异步网络上以无限的消息延迟达成共识。事实上，分布式系统始终会达成共识。这是可能的，因为虽然我们的网络是异步的，但我们可以对消息延迟建立合理的实际界限，并在超时后重试。因此，FLP 是最坏的情况，因此我将在第 12 章中讨论在分布式数据库中建立共识的算法。

Luckily, this is only a theoretical limitation, demonstrating it’s not possible to guarantee consensus will be reached with unbounded message delays on an asynchronous network. In reality, distributed systems reach consensus all the time. This is possible because while our networks are asynchronous, we can establish sensible practical bounds on message delays and retry after a timeout period. FLP is therefore a worst-case scenario, and as such I’ll discuss algorithms for establishing consensus in distributed databases in Chapter 12.

最后，我们应该注意拜占庭失败的问题。想象一下将两位将军的问题扩展到需要就进攻时间达成一致的N位将军。然而，在这种情况下，叛徒信使可能会改变攻击时间的值，或者叛将可能会向其他将军发送虚假信息。

Finally, we should note the issue of Byzantine failures. Imagine extending the Two Generals’ Problem to N generals who need to agree on a time to attack. However, in this scenario, traitorous messengers may change the value of the time of the attack, or a traitorous general may send false information to other generals.

这类恶意故障被称为拜占庭故障，在分布式系统中尤其险恶。幸运的是，我们在本书中讨论的系统通常位于受良好保护的、安全的企业网络和管理环境后面。这意味着我们实际上可以排除处理拜占庭错误。确实存在解决此类恶意行为的算法，如果您对实际示例感兴趣，请采取看看区块链共识机制和比特币。

This class of malicious failures are known as Byzantine faults and are particularly sinister in distributed systems. Luckily, the systems we discuss in this book typically live behind well-protected, secure enterprise networks and administrative environments. This means we can in practice exclude handling Byzantine faults. Algorithms that do address such malicious behaviors exist, and if you are interested in a practical example, take a look at blockchain consensus mechanisms and Bitcoin.

分布式系统中的时间

Time in Distributed Systems

分布式系统中的每个节点都有自己的内部时钟。如果每台机器上的所有时钟都完美的话通过同步，我们总是可以简单地比较跨节点事件的时间戳，以确定它们发生的精确顺序。如果这是现实，我将讨论的分布式系统的许多问题几乎都会消失。

Every node in a distributed system has its own internal clock. If all the clocks on every machine were perfectly synchronized, we could always simply compare the timestamps on events across nodes to determine the precise order they occurred in. If this were reality, many of the problems I’ll discuss with distributed systems would pretty much go away.

不幸的是，这种情况并非如此。由于环境因素，各个节点上的时钟会发生漂移温度或电压变化等条件。每台机器的漂移量各不相同，但每天 10-20 秒等数值并不少见。（或者用我家里现在的咖啡机，每天大约 5 分钟！）

Unfortunately, this is not the case. Clocks on individual nodes drift due to environmental conditions like changes in temperature or voltage. The amount of drift varies on every machine, but values such as 10–20 seconds per day are not uncommon. (Or with my current coffee machine at home, about 5 minutes per day!)

如果不加以控制，时钟漂移将使节点上的时间变得毫无意义——就像我的咖啡机上的时间，如果我不每隔几天纠正一次。为了解决这个问题，存在多种时间服务。时间服务代表准确的时间源，例如 GPS 或原子钟，可用于定期重置节点上的时钟，以纠正数据包交换、可变延迟数据网络上的漂移。

If left unchecked, clock drift would render the time on a node meaningless—like the time on my coffee machine if I don’t correct it every few days. To address this problem, a number of time services exist. A time service represents an accurate time source, such as a GPS or atomic clock, which can be used to periodically reset the clock on a node to correct for drift on packet-switched, variable-latency data networks.

使用最广泛的时间服务是网络时间协议 (NTP)，它提供跨越全球的按层次结构组织的时间服务器集合。根服务器在全球范围内约有 300 个，是最准确的。层次结构的下一级（大约 20,000 个）中的时间服务器会定期与根服务器的几毫秒内同步，在整个层次结构中依此类推，最多 15 级。全球有超过 175,000 个 NTP 服务器。

The most widely used time service is Network Time Protocol (NTP), which provides a hierarchically organized collection of time servers spanning the globe. The root servers, of which there are around 300 worldwide, are the most accurate. Time servers in the next level of the hierarchy (approximately 20,000) synchronize to within a few milliseconds of the root server periodically, and so on throughout the hierarchy, with a maximum of 15 levels. Globally, there are more than 175,000 NTP servers.

使用 NTP 协议，运行 NTP 客户端的应用程序中的节点可以同步到 NTP 服务器。节点上的时间是通过与一个或多个 NTP 服务器交换 UDP 消息来设置的。消息带有时间戳，并且通过消息交换来估计消息传输所花费的时间。这成为 NTP 使用的算法中的一个因素，用于确定客户端上的时间应重置为什么。一个简单的NTP配置如图3-9所示。在 LAN 上，机器可以在少量毫秒的精度内同步到 NTP 服务器。

Using the NTP protocol, a node in an application running an NTP client can synchronize to an NTP server. The time on a node is set by a UDP message exchange with one or more NTP servers. Messages are time stamped, and through the message exchange the time taken for message transit is estimated. This becomes a factor in the algorithm used by NTP to establish what the time on the client should be reset to. A simple NTP configuration is shown in Figure 3-9. On a LAN, machines can synchronize to an NTP server within a small number of milliseconds accuracy.

NTP 同步对我们的应用程序的一个有趣的影响是时钟的重置可以使本地节点时间向前或向后移动。NTP这意味着，如果我们的应用程序正在测量事件发生所需的时间（例如，计算事件响应时间），如果协议设置了本地时间，则事件的结束时间可能早于开始时间落后。

One interesting effect of NTP synchronization for our applications is that the resetting of the clock can move the local node time forward or backward. This means that if our application is measuring the time taken for events to occur (e.g., to calculate event response times), it is possible that the end time of the event may be earlier than the start time if the NTP protocol has set the local time backward.

图 3-9。说明如何使用 NTP 服务

事实上，一个计算节点有两个时钟。这些都是：

In fact, a compute node has two clocks. These are:

一天中的时间时钟

Time of day clock

这代表了数量自 1970 年 1 月 1 日午夜以来的毫秒数。在 Java 中，您可以使用 获取当前时间System.currentTimeMillis()。这是可以由 NTP 重置的时钟，因此如果它比 NTP 时间落后或提前很远，则可能会向前或向后跳跃。

This represents the number of milliseconds since midnight, January 1st 1970. In Java, you can get the current time using System.currentTimeMillis(). This is the clock that can be reset by NTP, and hence may jump forward or backward if it is a long way behind or ahead of NTP time.

单调时钟

Monotonic clock

这表示自过去未指定的点以来的时间量（以秒和纳秒为单位），例如上次系统重新启动的时间。它只会永远向前；然而，它也可能不是对经过时间的完全准确的测量，因为它在虚拟机挂起等事件期间停止。在 Java 中，您可以使用 获取当前单调时钟时间Sys⁠tem.nanoTime()。

This represents the amount of time (in seconds and nanoseconds) since an unspecified point in the past, such as the last time the system was restarted. It will only ever move forward; however, it again may not be a totally accurate measure of elapsed time because it stalls during an event such as virtual machine suspension. In Java, you can get the current monotonic clock time using Sys⁠tem.nanoTime().

应用程序可以使用 NTP 服务来确保系统中每个节点上的时钟紧密同步。应用程序通常会以一小时到一天的时间间隔重新同步时钟。这确保了时钟的值保持接近。尽管如此，如果应用程序确实需要精确地知道不同节点上发生的事件的顺序，时钟漂移将使这充满危险。

Applications can use an NTP service to ensure the clocks on every node in the system are closely synchronized. It’s typical for an application to resynchronize clocks on anything from a one hour to one day time interval. This ensures the clocks remain close in value. Still, if an application really needs to precisely know the order of events that occur on different nodes, clock drift is going to make this fraught with danger.

还有其他时间服务可以提供比 NTP 更高的精度。Chrony支持 NTP 协议，但提供比 NTP 更高的准确性和更大的可扩展性，这也是Facebook 采用它的原因。亚马逊通过在其数据中心安装 GPS 和原子钟构建了亚马逊时间同步服务。该服务免费向所有 AWS 客户开放。

There are other time services that provide higher accuracy than NTP. Chrony supports the NTP protocol but provides much higher accuracy and greater scalability than NTP—the reason it has been adopted by Facebook. Amazon has built the Amazon Time Sync Service by installing GPS and atomic clocks in its data centers. This service is available for free to all AWS customers.

这次讨论的要点是，我们的应用程序不能依赖不同节点上的事件时间戳来表示这些事件的实际顺序。即使时钟漂移一两秒，跨节点时间戳也变得毫无意义进行比较。这的影响当我们开始详细讨论分布式数据库时就会变得清楚。

The takeaway from this discussion is that our applications cannot rely on timestamps of events on different nodes to represent the actual order of these events. Clock drift even by a second or two makes cross-node timestamps meaningless to compare. The implications of this will become clear when we start to discuss distributed databases in detail.

总结和延伸阅读

Summary and Further Reading

本章涵盖了大量内容来解释分布式系统中通信和时间的一些基本特征。这些特征对于应用程序设计者和开发者来说非常重要。

This chapter has covered a lot of ground to explain some of the essential characteristics of communications and time in distributed systems. These characteristics are important for application designers and developers to understand.

本章应引起共鸣的关键问题如下：

The key issues that should resonate from this chapter are as follows:

1. 分布式系统中的通信可以透明地穿越许多不同类型的底层物理网络，包括 WiFi、无线、WAN 和 LAN。因此，通信延迟变化很大，并受到节点之间的物理距离、物理网络属性和瞬时网络拥塞的影响。在大规模情况下，应用程序组件之间的延迟应该尽可能地最小化（当然，在物理定律的范围内）。

2. Communications in distributed systems can transparently traverse many different types of underlying physical networks, including WiFi, wireless, WANs, and LANs. Communication latencies are hence highly variable, and influenced by the physical distance between nodes, physical network properties, and transient network congestion. At large scale, latencies between application components are something that should be minimized as much as possible (within the laws of physics, of course).

3. 互联网协议栈通过 IP 和 TCP 协议的组合确保跨异构网络的可靠通信。由于网络通信结构和路由器故障导致节点不可用，以及单个节点故障，通信可能会失败。您的代码将经历各种 TCP/IP 开销，例如连接建立以及网络故障发生时的错误。因此，了解 IP 套件的基础知识对于设计和调试非常重要。

4. The Internet Protocol stack ensures reliable communications across heterogeneous networks through a combination of the IP and TCP protocols. Communications can fail due to network communications fabric and router failures that make nodes unavailable, as well as individual node failure. Your code will experience various TCP/IP overheads, for example, for connection establishment, and errors when network failures occur. Hence, understanding the basics of the IP suite is important for design and debugging.

5. RMI/RPC 技术构建了 TCP/IP 层，为客户端/服务器通信提供抽象，从而镜像本地方法/过程调用。然而，这些更抽象的编程方法仍然需要能够适应网络问题，例如故障和重传。这在服务器上改变状态的应用程序 API 中最为明显，并且必须设计为幂等的。

6. RMI/RPC technologies build the TCP/IP layer to provide abstractions for client/server communications that mirror making local method/procedure calls. However, these more abstract programming approaches still need to be resilient to network issues such as failures and retransmissions. This is most apparent in application APIs that mutate state on the server, and must be designed to be idempotent.

7. 在异步网络上，在存在崩溃故障的情况下，不可能在有限时间内就多个节点之间的状态达成一致或共识。幸运的是，真实的网络，尤其是 LAN，速度快且可靠，这意味着我们可以设计出在实践中达成共识的算法。当我们讨论分布式数据库时，我将在本书的第三部分中介绍这些内容。

8. Achieving agreement, or consensus on state across multiple nodes in the presence of crash faults is not possible in bounded time on asynchronous networks. Luckily, real networks, especially LANs, are fast and mostly reliable, meaning we can devise algorithms that achieve consensus in practice. I’ll cover these in Part III of the book when we discuss distributed databases.

9. 应用程序中的节点不存在可以依赖的可靠的全局时间源来同步其行为。各个节点上的时钟各不相同，无法用于有意义的比较。这意味着应用程序无法有意义地比较不同节点上的时钟来确定事件的顺序。

10. There is no reliable global time source that nodes in an application can rely upon to synchronize their behavior. Clocks on individual nodes vary and cannot be used for meaningful comparisons. This means applications cannot meaningfully compare clocks on different nodes to determine the order of events.

这些问题将贯穿本书其余部分的讨论。分布式系统中采用的许多独特问题和解决方案都源于这些基本原理。他们无法逃脱！

These issues will pervade the discussions in the rest of this book. Many of the unique problems and solutions that are adopted in distributed systems stem from these fundamentals. There’s no escaping them!

George Coulouris 等人的《分布式系统：概念与设计》第 5 版是对分布式系统各个方面进行更详细、更理论性介绍的绝佳来源。（皮尔逊，2001）。

An excellent source for more detailed, more theoretical coverage of all aspects of distributed systems is George Coulouris et al., Distributed Systems: Concepts and Design, 5th ed. (Pearson, 2001).

同样，对于计算机网络，您将在 James Kurose 和 Keith Ross 的《计算机网络：自上而下的方法》第 7版中找到您想了解的所有信息，毫无疑问还有更多信息。（皮尔逊，2017）。

Likewise for computer networking, you’ll find out all you wanted to know and no doubt more in James Kurose and Keith Ross’s Computer Networking: A Top-Down Approach, 7th ed. (Pearson, 2017).

为什么要并发？

Why Concurrency?

想象一下一家繁忙的咖啡店。如果每个人都点一杯简单的咖啡，那么咖啡师可以快速、一致地提供咖啡每喝一杯。突然，你前面的人点了一杯大豆、香草、无糖、四杯冰镇啤酒。排队的每个人都叹了口气，开始阅读他们的社交媒体。两分钟后队伍就排到了门外。

Think of a busy coffee shop. If everyone orders a simple coffee, then the barista can quickly and consistently deliver each drink. Suddenly, the person in front of you orders a soy, vanilla, no sugar, quadruple-shot iced brew. Everyone in line sighs and starts reading their social media. In two minutes the line is out of the door.

Web 应用程序中的处理请求类似于我们的咖啡示例。在一家咖啡店，我们寻求新咖啡师的帮助，在不同的机器上同时制作咖啡，以控制生产线长度并快速为顾客提供服务。在软件中，为了使应用程序具有响应能力，我们需要以某种方式以重叠的方式处理服务器中的请求，同时处理请求。

Processing requests in web applications is analogous to our coffee example. In a coffee shop, we enlist the help of a new barista to simultaneously make coffees on a different machine to keep the line length in control and serve customers quickly. In software, to make applications responsive, we need to somehow process requests in our server in an overlapping manner, handling requests concurrently.

在过去的计算时代，每个 CPU 在任何时刻都只能执行一条机器指令。如果我们的服务器应用程序在这样的 CPU 上运行，为什么我们需要构建我们的软件系统以可能同时执行多个指令？这一切似乎都有点毫无意义。

In the good old days of computing, each CPU was only able to execute a single machine instruction at any instant. If our server application runs on such a CPU, why do we need to structure our software systems to potentially execute multiple instructions concurrently? It all seems slightly pointless.

实际上有一个很好的理由。事实上，每个程序不仅仅执行机器指令。例如，当程序尝试读取文件或在网络上发送消息时，它必须与 CPU 外围的硬件子系统（磁盘、网卡）进行交互。从磁硬盘读取数据大约需要 10 毫秒 (ms)。在此期间，程序必须等待数据可供处理。

There is actually a very good reason. Virtually every program does more than just execute machine instructions. For example, when a program attempts to read from a file or send a message on the network, it must interact with the hardware subsystem (disk, network card) that is peripheral to the CPU. Reading data from a magnetic hard disk takes around 10 milliseconds (ms). During this time, the program must wait for the data to be available for processing.

现在，即使是古老的 CPU，例如1988 年的 Intel 80386，也可以每秒执行超过 1000 万条指令 (mips) 。10 毫秒是百分之一秒。我们的 80386 在百分之一秒内可以执行多少条指令？算一算。（提示——很多！）事实上，浪费了很多处理能力。

Now, even an ancient CPU such as a 1988 Intel 80386 can execute more than 10 million instructions per second (mips). 10 ms is one hundredth of a second. How many instructions could our 80386 execute in a hundredth second? Do the math. (Hint—it’s a lot!) A lot of wasted processing capacity, in fact.

这就是 Linux 等操作系统可以在单个 CPU 上运行多个程序的方式。当一个程序等待 I/O 事件时，操作系统会调度另一个程序来执行。通过显式地构造我们的软件以具有可以并行执行的多个活动，操作系统可以安排有工作要做的任务，而其他任务则等待 I/O。我们将在本章后面更详细地了解它如何与 Java 一起工作。

This is how operating systems such as Linux can run multiple programs on a single CPU. While one program is waiting for an I/O event, the operating system schedules another program to execute. By explicitly structuring our software to have multiple activities that can be executed in parallel, the operating system can schedule tasks that have work to do while others wait for I/O. We’ll see in more detail how this works with Java later in this chapter.

2001 年，IBM 推出了世界上第一个多核处理器，即具有两个CPU 的芯片 —请参阅图 4-1的简化说明。如今，就连我的笔记本电脑也有 16 个 CPU，即俗称的“核心”。使用多核芯片，可以执行具有多个并行活动的软件系统每个核心上同时运行，最多可达可用核心的数量。这样，我们就可以充分利用多核芯片上的处理资源，从而提高我们应用程序的处理能力。

In 2001, IBM introduced the world’s first multicore processor, a chip with two CPUs—see Figure 4-1 for a simplified illustration. Today, even my laptop has 16 CPUs, or “cores,” as they are commonly known. With a multicore chip, a software system that is structured to have multiple parallel activities can be executed concurrently on each core, up to the number of available cores. In this way, we can fully utilize the processing resources on a multicore chip, and thus increase our application’s processing capacity.

图 4-1。多核处理器的简化视图

将软件系统构建为并发活动的主要方法是使用线程。几乎每一个编程语言有自己的线程机制。所有这些机制的底层语义都是相似的——主流使用的主要线程模型只有少数——但显然语法因语言而异。在下面的部分中，我将解释 Java 中如何支持线程，以及如何设计我们的程序，使其在并行执行时安全（即正确）和高效。有了这些知识，跳入其他语言支持的并发特性应该不会太困难。

The primary way to structure a software system as concurrent activities is to use threads. Virtually every programming language has its own threading mechanism. The underlying semantics of all these mechanisms are similar—there are only a few primary threading models in mainstream use—but obviously the syntax varies by language. In the following sections, I’ll explain how threads are supported in Java, and how we need to design our programs to be safe (i.e., correct) and efficient when executing in parallel. Armed with this knowledge, leaping into the concurrency features supported in other languages shouldn’t be too arduous.

线程数

Threads

每个软件进程都有一个默认执行线程。这是操作系统在调度进程执行时管理的线程。例如，在 Java 中，main()您指定为代码入口点的函数定义了该线程的行为。该单个线程可以访问程序的环境和资源，例如打开的文件句柄和网络连接。当程序调用代码中实例化的对象中的方法时，程序的运行时堆栈用于传递参数和管理变量范围。标准编程语言运行时的东西，我们都知道并且喜欢。这是一个顺序过程。

Every software process has a single thread of execution by default. This is the thread that the operating system manages when it schedules the process for execution. In Java, for example, the main() function you specify as the entry point to your code defines the behavior of this thread. This single thread has access to the program’s environment and resources such as open file handles and network connections. As the program calls methods in objects instantiated in the code, the program’s runtime stack is used to pass parameters and manage variable scopes. Standard programming language runtime stuff, that we all know and love. This is a sequential process.

在您的系统中，您可以使用编程语言功能来创建和执行其他线程。每个线程都是一个独立的执行序列，并且有自己的运行时堆栈来管理本地对象创建和方法调用。每个线程还可以访问进程的全局数据和环境。该方案的简单描述如图 4-2所示。

In your systems, you can use programming language features to create and execute additional threads. Each thread is an independent sequence of execution and has its own runtime stack to manage local object creation and method calls. Each thread also has access to the process’ global data and environment. A simple depiction of this scheme is shown in Figure 4-2.

图 4-2。比较单线程和多线程进程

在Java中，我们可以使用a来定义线程实现Runnable接口并定义run()方法的类。让我们看一个简单的例子：

In Java, we can define a thread using a class that implements the Runnable interface and defines the run() method. Let’s look at a simple example:

类 NamingThread 实现Runnable{

私有字符串名称；
       
公共命名线程（字符串线程名）{
      名称 = 线程名称;
           System.out.println("构造函数调用：" + threadName) ;
       }
        
       公共无效run(){
      //显示该线程的信息
           System.out.println("运行名为：" + name);
           System.out.println(name + " : " + Thread.currentThread());
           // 现在终止....
    }
}

class NamingThread implements Runnable {

private String name;
       
public NamingThread(String threadName) {
      name = threadName ;
           System.out.println("Constructor called: " + threadName) ;
       }
        
       public void run() {
      //Display info about this  thread
           System.out.println("Run called : " + name);
           System.out.println(name + " : " + Thread.currentThread());
           // and now terminate  ....
    }
}

为了执行线程，我们需要Thread使用 our 的实例构造一个对象Runnable，并调用该start()方法以在其自己的执行上下文中调用代码。这将在下一个代码示例中显示，并以粗体文本显示运行代码的输出。请注意，此示例有两个线程：main()线程和NamingThread. 主线程启动NamingThread异步执行的 ，然后等待 1 秒，以便我们的run()方法有NamingThread足够的时间完成：

To execute the thread, we need to construct a Thread object using an instance of our Runnable and call the start() method to invoke the code in its own execution context. This is shown in the next code example, along with the output of running the code in bold text. Note this example has two threads: the main() thread and the NamingThread. The main thread starts the NamingThread, which executes asynchronously, and then waits for 1 second to give our run() method in NamingThread ample time to complete:

公共静态无效主（字符串[] args）{
      
  NamingThread name0 = new NamingThread("我的第一个线程");
    
  //创建线程
  线程 t0 = new Thread(name0);
    
  // 启动线程
  t0.start();
      
  //延迟主线程一秒（1000毫秒）
  尝试 {
    Thread.currentThread().sleep(1000);
  } catch (InterruptedException e) {}

      //显示主线程信息并终止
      System.out.println(Thread.currentThread());
    
}

===执行输出===
构造函数调用：我的第一个线程
运行名为：我的第一个线程
我的第一个线程：Thread[Thread-0,5,main]
线程[主,5,主]

public static void main(String[] args) {
      
  NamingThread name0 = new NamingThread("My first thread");
    
  //Create the thread
  Thread t0 = new Thread (name0);
    
  // start the threads
  t0.start();
      
  //delay the main thread for a second (1000 milliseconds)
  try {
    Thread.currentThread().sleep(1000);
  } catch (InterruptedException e) {}

      //Display info about the main thread and terminate
      System.out.println(Thread.currentThread());
    
}

===EXECUTION OUTPUT===
Constructor called: My first thread
Run called : My first thread
My first thread : Thread[Thread-0,5,main]
Thread[main,5,main]

为了便于说明，我们也称静态currentThread()方法，返回一个字符串，其中包含：

For illustration, we also call the static currentThread() method, which returns a string containing:

• 系统生成的线程标识符。

• The system-generated thread identifier.

• 线程优先级，所有线程默认为 5。稍后我们将介绍线程优先级。

• The thread priority, which by default is 5 for all threads. We’ll cover thread priorities later.

• 父线程的标识符——在此示例中，两个父线程都是main thread.

• The identifier of the parent thread—in this example both parent threads are the main thread.

请注意，要实例化线程，我们调用start()方法，而不是run()我们在Runnable. 该start()方法包含内部系统魔法，用于为要执行的单独线程创建执行上下文。如果我们run()直接调用，代码将会执行，但不会创建新的线程。该run()方法将作为main线程的一部分执行，就像您了解和喜爱的任何其他 Java 方法调用一样。您仍然会有单线程代码。

Note that to instantiate a thread, we call the start() method, not the run() method we define in the Runnable. The start() method contains the internal system magic to create the execution context for a separate thread to execute. If we call run() directly, the code will execute, but no new thread will be created. The run() method will execute as part of the main thread, just like any other Java method invocation that you know and love. You will still have a single-threaded code.

在示例中，我们使用sleep()暂停线程的执行main并确保它不会在NamimgThread. 这种方法，即通过延迟绝对时间段（示例中为 1 秒）来协调两个线程，并不是一种非常稳健的机制。如果由于某种原因（CPU 速度较慢、磁盘读取延迟较长、方法中额外的复杂逻辑）我们的线程没有在预期的时间范围内终止怎么办？在这种情况下，main将首先终止——这不是我们的意图。一般来说，如果您使用绝对时间来进行线程协调，那么您就做错了。几乎总是。99.99999% 的时间都是如此。

In the example, we use sleep() to pause the execution of the main thread and make sure it does not terminate before the NamimgThread. This approach, namely coordinating two threads by delaying for an absolute time period (1 second in the example) is not a very robust mechanism. What if for some reason—a slower CPU, a long delay reading disk, additional complex logic in the method—our thread doesn’t terminate in the expected timeframe? In this case, main will terminate first—this is not what we intend. In general, if you are using absolute times for thread coordination, you are doing it wrong. Almost always. Like 99.99999% of the time.

使用该方法是一种简单而可靠的机制，可以让一个线程等待另一个线程完成其工作join()。我们可以将try-catch上面示例中的块替换为：

A simple and robust mechanism for one thread to wait until another has completed its work is to use the join() method. We could replace the try-catch block in the above example with:

t0.join();

t0.join();

此方法导致调用线程（在本例中为main）阻塞，直到 引用的线程t0终止。如果引用的线程在调用 之前已终止join()，则方法调用立即返回。通过这种方式，我们可以协调或同步多个线程的行为。多线程的同步实际上是其余的主要焦点本章的。

This method causes the calling thread (in this case, main) to block until the thread referenced by t0 terminates. If the referenced thread has terminated before the call to join(), then the method call returns immediately. In this way we can coordinate, or synchronize, the behavior of multiple threads. Synchronization of multiple threads is in fact the major focus of the rest of this chapter.

线程执行顺序

Order of Thread Execution

系统调度程序（在 Java 中，这就是在 Java 虚拟机 [JVM] 中）控制线程执行的顺序。从程序员的角度来看，执行顺序是不确定的。习惯这个词，我会经常使用它。不确定性的概念是理解多线程代码的基础。

The system scheduler (in Java, this lives in the Java virtual machine [JVM]) controls the order of thread execution. From the programmer’s perspective, the order of execution is nondeterministic. Get used to that term, I’ll use it a lot. The concept of nondeterminism is fundamental to understanding multithreaded code.

我将通过前面的示例来说明这一点NamingThread。NamingThread我将创建并启动一些，而不是创建一个。三、其实如下面的代码示例所示。同样，运行代码的示例输出在代码本身下方以粗体文本显示：

I’ll illustrate this by building on the earlier NamingThread example. Instead of creating a single NamingThread, I’ll create and start up a few. Three, in fact, as shown in the following code example. Again, sample output from running the code is in bold text beneath the code itself:

      NamingThread name0 = new NamingThread("thread0");
      NamingThread name1 = new NamingThread("线程1");
      NamingThread name2 = new NamingThread("thread2");
    
      //创建线程
      线程 t0 = new Thread(name0);
      线程 t1 = new Thread(name1);
      线程 t2 = new Thread(name2);    
      
      // 启动线程
      t0.start();  
      t1.start();  
      t2.start();  

===EXECUTION OUTPUT===
Run called : thread0
thread0 : Thread[Thread-0,5,main]  
Run called : thread2  
Run called : thread1
thread1 : Thread[Thread-1,5,main]  
thread2 : Thread[Thread-2,5,main]
Thread[main,5,main]

      NamingThread name0 = new NamingThread("thread0");
      NamingThread name1 = new NamingThread("thread1");
      NamingThread name2 = new NamingThread("thread2");
    
      //Create the threads
      Thread t0 = new Thread (name0);
      Thread t1 = new Thread (name1);
      Thread t2 = new Thread (name2);    
      
      // start the threads
      t0.start();  
      t1.start();  
      t2.start();  

===EXECUTION OUTPUT===
Run called : thread0
thread0 : Thread[Thread-0,5,main]  
Run called : thread2  
Run called : thread1
thread1 : Thread[Thread-1,5,main]  
thread2 : Thread[Thread-2,5,main]
Thread[main,5,main]

显示的输出只是一次执行的示例。您可以看到代码按顺序启动了三个线程，即t0、t1和t2（请参阅）。查看输出，我们看到线程t0在其他线程开始之前完成（请参阅）。接下来调用t2的方法（请参阅），然后调用t1的方法，即使t1在t2之前启动。然后，线程t1在t2之前运行完成（请参阅），并最终主线程和程序终止。run()run()

The output shown is a sample from just one execution. You can see the code starts three threads sequentially, namely t0, t1, and t2 (see ). Looking at the output, we see thread t0 completes (see ) before the others start. Next t2’s run() method is called (see ) followed by t1’s run() method, even though t1 was started before t2. Thread t1 then runs to completion (see ) before t2, and eventually the main thread and the program terminate.

这只是一种可能的执行顺序。如果我们再次运行这个程序，我们几乎肯定会看到不同的执行跟踪。这是因为 JVM 调度程序正在决定执行哪个线程以及执行多长时间。简而言之，一旦调度程序为某个线程提供了 CPU 上的执行时隙，它就可以在指定时间段后中断该线程并调度另一个线程运行。这种中断称为抢占。抢占确保每个线程都有机会取得进展。因此，线程独立且异步地运行直到完成，并且调度程序根据调度算法决定运行哪个线程。

This is just one possible order of execution. If we run this program again, we will almost certainly see a different execution trace. This is because the JVM scheduler is deciding which thread to execute, and for how long. Put very simply, once the scheduler has given a thread an execution time slot on a CPU, it can interrupt the thread after a specified time period and schedule another one to run. This interruption is known as preemption. Preemption ensures each thread is given an opportunity to make progress. Hence the threads run independently and asynchronously until completion, and the scheduler decides which thread runs when based on a scheduling algorithm.

线程调度的内容远不止这些，我将在本章后面解释使用的基本调度算法。目前，这对程序员来说有一个重大影响：无论线程执行的顺序如何（您无法控制），您的代码都应该产生正确的结果结果。听起来很容易？请继续阅读。

There’s more to thread scheduling than this, and I’ll explain the basic scheduling algorithm used later in this chapter. For now, there is a major implication for programmers; regardless of the order of thread execution—which you don’t control—your code should produce correct results. Sounds easy? Read on.

线程问题

Problems with Threads

并发编程的基本问题是协调多个线程的执行，以便无论它们以什么顺序执行，它们都会产生正确的答案。鉴于线程可以不确定地启动和抢占，任何中等复杂的程序本质上都将具有无限数量的可能的执行顺序。这些系统不容易测试。

The basic problem in concurrent programming is coordinating the execution of multiple threads so that whatever order they are executed in, they produce the correct answer. Given that threads can be started and preempted nondeterministically, any moderately complex program will have essentially an infinite number of possible orders of execution. These systems aren’t easy to test.

所有并发程序都需要避免两个基本问题。这些是竞争条件和死锁，这些主题将在接下来的两小节中介绍。

There are two fundamental problems that all concurrent programs need to avoid. These are race conditions and deadlocks, and these topics are covered in the next two subsections.

竞赛条件

Race Conditions

线程的非确定性执行意味着组成线程的代码语句：

Nondeterministic execution of threads implies that the code statements that comprise the threads:

• 将按照每个线程中的定义顺序执行。

• Will execute sequentially as defined within each thread.

• 可以跨线程以任意顺序重叠。这是因为每个线程执行槽执行的语句数量是由调度程序确定的。

• Can be overlapped in any order across threads. This is because the number of statements that are executed for each thread execution slot is determined by the scheduler.

因此，当执行多个线程时在单个处理器上，它们的执行是交错的。CPU 从一个线程执行一些步骤，然后从另一个线程执行一些步骤，依此类推。如果我们在多核 CPU 上执行，则每个核心可以执行一个线程。然而，每个线程执行的语句仍然以不确定的方式交错。

Hence, when many threads are executed on a single processor, their execution is interleaved. The CPU executes some steps from one thread, then performs some steps from another, and so on. If we are executing on a multicore CPU, then we can execute one thread per core. The statements of each thread execution are still however interleaved in a nondeterministic manner.

现在，如果每个线程只是做自己的事情并且完全独立，那么这不是问题。每个线程都会执行直到终止，如我们的简单Naming​Thread示例所示。这东西简直就是小菜一碟！为什么这些线程的东西注定是复杂的？

Now, if every thread simply does its own thing and is completely independent, this is not a problem. Each thread executes until it terminates, as in our trivial Naming​Thread example. This stuff is a piece of cake! Why are these thread things meant to be complex?

不幸的是，完全独立的线程并不是大多数多线程系统的行为方式。如果您回头参考图 4-2，您将看到多个线程共享进程内的全局数据。在 Java 中，这既是全局数据又是静态数据。

Unfortunately, totally independent threads are not how most multithreaded systems behave. If you refer back to Figure 4-2, you will see that multiple threads share the global data within a process. In Java this is both global and static data.

线程可以使用共享数据结构来协调它们的工作并跨线程传达状态。例如，我们可能有处理来自 Web 客户端的请求的线程，每个请求一个线程。我们还希望保留每天处理的请求总数。当线程完成请求时，它会增加RequestCounter所有线程在每次请求后共享和更新的全局对象。最终，我们知道处理了多少请求。确实是一个简单而优雅的解决方案。也许会？

Threads can use shared data structures to coordinate their work and communicate status across threads. For example, we may have threads handling requests from web clients, one thread per request. We also want to keep a running total of how many requests we process each day. When a thread completes a request, it increments a global RequestCounter object that all threads share and update after each request. At the end of the day, we know how many requests were processed. A simple and elegant solution indeed. Well, maybe?

下面的代码显示了一个非常简单的实现，它模仿了请求计数器示例场景。它创建 50,000 个线程来更新共享计数器。请注意，为了简洁起见，我们使用 lambda 函数来创建线程，并使用（非常糟糕的主意）5 秒的延迟来main允许线程完成：¹

The code below shows a very simple implementation that mimics the request counter example scenario. It creates 50,000 threads to update a shared counter. Note we use a lambda function for brevity to create the threads, and a (really bad idea) 5-second delay in main to allow the threads to finish:¹

公共类请求计数器{
  最终静态私有 int NUMTHREADS = 50000；
  私有整数计数 = 0；
    
  公共无效公司（）{
    计数++；
  }
    
  公共 int getVal() {
    返回这个.count；
  }
    
  公共静态无效主（字符串[] args）抛出InterruptedException {
    最终 RequestCounter 计数器 = new RequestCounter();
          
    for (int i = 0; i < NUMTHREADS; i++) {
      // lambda 可运行创建
      可运行线程 = () -> {counter.inc(); };
        新线程(线程).start();
    }
          
    线程.sleep(5000);
    System.out.println("值应该是 " + NUMTHREADS + "是：" + counter.getVal());
  }
}

public class RequestCounter {
  final static private int NUMTHREADS = 50000;
  private int count = 0;
    
  public  void inc() {
    count++;
  }
    
  public int getVal() {
    return this.count;
  }
    
  public static void main(String[] args) throws InterruptedException {
    final RequestCounter counter = new RequestCounter();
          
    for (int i = 0; i < NUMTHREADS; i++) {
      // lambda runnable creation 
      Runnable thread = () -> {counter.inc(); };
        new Thread(thread).start();
    }
          
    Thread.sleep(5000);
    System.out.println("Value should be " + NUMTHREADS + "It is: " +     counter.getVal());
  }
}

您可以在家做的就是从本书的 GitHub 存储库克隆此代码，运行此代码几次，然后看看会得到什么结果。在 10 次处决中，我的平均值是 49,995。我一次也没有得到50,000的正确答案。诡异的。

What you can do at home is clone this code from the book’s GitHub repo, run this code a few times, and see what results you get. In 10 executions my mean was 49,995. I didn’t once get the correct answer of 50,000. Weird.

为什么？

Why?

答案在于如何在机器上执行抽象的高级编程语言语句（本例中为 Java）。在此示例中，执行增量计数器，CPU 必须：

The answer lies in how abstract, high-level programming language statements, in Java in this case, are executed on a machine. In this example, to perform an increment of a counter, the CPU must:

• 将当前值加载到寄存器中。

• Load the current value into a register.

• 增加寄存器值。

• Increment the register value.

• 将结果写回原始内存位置。

• Write the results back to the original memory location.

这个简单的增量实际上是三个机器级操作的序列。

This simple increment is actually a sequence of three machine-level operations.

如图4-3所示，在机器级别，这三个操作是独立的，不被视为单个原子操作。通过原子，我们的意思是一个不能被中断的操作，因此一旦开始就会运行到完成。

As Figure 4-3 shows, at the machine level these three operations are independent and not treated as a single atomic operation. By atomic, we mean an operation that cannot be interrupted and hence once started will run to completion.

随着增量操作在机器级别上不是原子的，一个线程可以将计数器值从内存加载到 CPU 寄存器中，但在将递增的值写回之前，调度程序会抢占该线程并允许另一个线程启动。该线程从内存加载计数器的旧值并写回递增的值。最终，原始线程再次执行并写回其增量值，该值恰好与内存中已有的值相同。

As the increment operation is not atomic at the machine level, one thread can load the counter value into a CPU register from memory, but before it writes the incremented value back, the scheduler preempts the thread and allows another thread to start. This thread loads the old value of the counter from memory and writes back the incremented value. Eventually the original thread executes again and writes back its incremented value, which just happens to be the same as what is already in memory.

这意味着我们丢失了更新。从上面对计数器代码的 10 次测试中，我们发现这种情况平均以 50,000 次增量发生 5 次。因此，此类事件很少见，但即使发生千万分之一，你仍然会得到不正确的结果。

This means we’ve lost an update. From our 10 tests of the counter code above, we see this is happening on average 5 times in 50,000 increments. Hence such events are rare, but even if it happens 1 time in 10 million, you still have an incorrect result.

图 4-3。增量在机器级别不是原子的

当我们以这种方式丢失更新时，称为竞争条件。每当多个线程对某些共享状态（在本例中是一个简单的计数器）进行更改时，就会出现竞争条件。本质上，不同的线程交错可以产生不同的结果。

When we lose updates in this manner, it is called a race condition. Race conditions can occur whenever multiple threads make changes to some shared state, in this case a simple counter. Essentially, different interleavings of the threads can produce different results.

竞态条件是阴险、邪恶的错误，因为它们的发生通常很少见，并且很难检测到，因为大多数时候答案都是正确的。尝试使用 1,000 个线程而不是 50,000 个线程运行多线程计数器代码示例，您将看到它的实际效果。我十分之九得到了正确答案。

Race conditions are insidious, evil errors, because their occurrence is typically rare, and they can be hard to detect as most of the time the answer will be correct. Try running the multithreaded counter code example with 1,000 threads instead of 50,000, and you will see this in action. I got the correct answer nine times out of ten.

所以，这种情况可以概括为“相同的代码，偶尔会出现不同的结果”。就像我说的，竞争条件是邪恶的！幸运的是，如果采取一些预防措施，根除它们是很简单的。

So, this situation can be summarized as “same code, occasionally different results.” Like I said, race conditions are evil! Luckily, eradicating them is straightforward if you take a few precautions.

关键是识别和保护关键部分。关键部分是更新共享数据结构的代码，因此如果被多个线程访问，则必须以原子方式执行。递增共享计数器的示例是临界区的一个示例。另一个是从列表中删除一个项目。我们需要删除列表的头节点，并将对列表头的引用从删除的节点移动到列表中的下一个节点。这两个操作都必须以原子方式执行，以保持列表的完整性。这是一个关键部分。

The key is to identify and protect critical sections. A critical section is a section of code that updates shared data structures and hence must be executed atomically if accessed by multiple threads. The example of incrementing a shared counter is an example of a critical section. Another is removing an item from a list. We need to delete the head node of the list and move the reference to the head of the list from the removed node to the next node in the list. Both operations must be performed atomically to maintain the integrity of the list. This is a critical section.

在Java中，synchronized关键字定义一个关键部分。如果用于修饰一个方法，那么当多个线程尝试在同一个共享对象上调用该方法时，只有一个线程被允许进入临界区。所有其他线程都会阻塞，直到线程退出同步方法，此时调度程序选择下一个线程来执行关键部分。我们说临界区的执行是串行的，因为一次只有一个线程可以执行其中的代码。

In Java, the synchronized keyword defines a critical section. If used to decorate a method, then when multiple threads attempt to call that method on the same shared object, only one is permitted to enter the critical section. All others block until the thread exits the synchronized method, at which point the scheduler chooses the next thread to execute the critical section. We say the execution of the critical section is serialized, as only one thread at a time can be executing the code inside it.

因此，要修复反例，您只需将该inc()方法标识为关键部分并使其成为同步方法，即：

To fix the counterexample, you therefore just need to identify the inc() method as a critical section and make it a synchronized method, i.e.:

同步公共无效公司（）{
    计数++；
  }

synchronized public void inc() {
    count++;
  }

您可以根据需要多次测试。你总会得到正确的答案。更正式地说，这意味着调度程序向我们抛出的任何线程交错都将始终产生正确的结果。

Test it out as many times as you like. You’ll always get the correct answer. Slightly more formally, this means any interleaving of the threads that the scheduler throws at us will always produce the correct results.

该synchronized关键字还可以应用于方法内的语句块。例如，我们可以将上面的例子重写为：

The synchronized keyword can also be applied to blocks of statements within a method. For example, we could rewrite the above example as:

公共无效公司（）{
        同步（这个）{
           计数++；   
        }
}

public void inc() {
        synchronized(this){
           count++;   
        }
}

在幕后，每个 Java 对象都有一个监视器锁（有时称为内在锁）作为其运行时表示的一部分。监视器就像长途巴士上的卫生间一样，一次只允许（并且应该！）一个人进入，并且门锁在使用时阻止其他人进入。

Underneath the covers, every Java object has a monitor lock, sometimes known as an intrinsic lock, as part of its runtime representation. The monitor is like the bathroom on a long-distance bus—only one person is allowed to (and should!) enter at once, and the door lock stops others from entering when in use.

在我们完全卫生的 Java 运行时环境中，线程必须获取监视器锁才能进入同步方法或同步语句块。任何时候只有一个线程可以拥有锁，因此执行是串行的。基本上，这就是 Java 和类似语言实现临界区的方式。

In our totally sanitary Java runtime environment, a thread must acquire the monitor lock to enter a synchronized method or synchronized block of statements. Only one thread can own the lock at any time, and hence execution is serialized. This, very basically, is how Java and similar languages implement critical sections.

根据经验，您应该使关键部分尽可能小，以便最大限度地减少序列化代码。这会对性能和可扩展性产生积极影响。稍后我将回到这个主题，但我实际上再次谈论阿姆达尔定律，如第 2 章中介绍的那样。同步块是 Amdahl 所描述的系统的序列化部分，它们执行的时间越长，那么系统可扩展性的潜力就越小。

As a rule of thumb, you should keep critical sections as small as possible so that the serialized code is minimized. This can have positive impacts on performance and hence scalability. I’ll return to this topic later, but I’m really talking about Amdahl’s law again, as introduced in Chapter 2. Synchronized blocks are the serialized parts of a system as described by Amdahl, and the longer they execute for, then the less potential we have for system scalability.

僵局

Deadlocks

为了确保多线程代码中的正确结果，我解释说我们必须限制固有的不确定性序列化对关键部分的访问。这避免了竞争条件。但是，如果我们不小心，我们可能会编写过多限制非确定性的代码，从而导致程序停止。并且永远不会继续。这正式称为死锁。

To ensure correct results in multithreaded code, I explained that we have to restrict the inherent nondeterminism to serialize access to critical sections. This avoids race conditions. However, if we are not careful, we can write code that restricts nondeterminism so much that our program stops. And never continues. This is formally known as a deadlock.

当两个或多个线程永远被阻塞并且没有一个线程可以继续时，就会发生死锁。当线程需要独占访问一组共享资源并且线程以不同的顺序获取锁时，就会发生这种情况。下面的示例对此进行了说明，其中两个线程需要对临界区 A 和 B 进行独占访问。线程 1获取临界区 A 的锁，线程 2 获取临界区 B 的锁。然后，两个线程都将永远阻塞，因为它们无法获取他们继续需要的锁。

A deadlock occurs when two or more threads are blocked forever, and none can proceed. This happens when threads need exclusive access to a shared set of resources and the threads acquire locks in different orders. This is illustrated in the example below in which two threads need exclusive access to critical sections A and B. Thread 1 acquires the lock for critical section A, and thread 2 acquires the lock for critical section B. Both then block forever as they cannot acquire the locks they need to continue.

两个线程通过同步块共享对两个共享变量的访问：

Two threads sharing access to two shared variables via synchronized blocks:

• 线程1：进入临界区A。

• Thread 1: enters critical section A.

• 线程2：进入临界区B。

• Thread 2: enters critical section B.

• 线程 1：在进入临界区 B 时阻塞。

• Thread 1: blocks on entry to critical section B.

• 线程 2：在进入临界区 A 时阻塞。

• Thread 2: blocks on entry to critical section A.

• 两个线程都永远等待。

• Both threads wait forever.

死锁，也称为致命的拥抱，导致程序停止。不需要生动的想象力就可以意识到这可能会导致各种不良结果。当我的自动驾驶汽车载我去酒吧时，我很高兴地发短信。突然，车辆代码陷入僵局。不会有好结局的。

A deadlock, also known as a deadly embrace, causes a program to stop. It doesn’t take a vivid imagination to realize that this can cause all sorts of undesirable outcomes. I’m happily texting away while my autonomous vehicle drives me to the bar. Suddenly, the vehicle code deadlocks. It won’t end well.

死锁发生在比上面的简单示例更微妙的情况下。典型的例子是哲学家就餐问题。故事是这样的。

Deadlocks occur in more subtle circumstances than the simple example above. The classic example is the dining philosophers problem. The story goes like this.

五位哲学家围坐在一起桌子。作为哲学家，他们花费大量时间进行深入思考。在深度思考的间隙，他们通过吃面前的一盘食物来补充大脑功能。因此，哲学家要么在吃饭，要么在思考，或者在这两种状态之间转换。

Five philosophers sit around a shared table. Being philosophers, they spend a lot of time thinking deeply. In between bouts of deep thinking, they replenish their brain function by eating from a plate of food that sits in front of them. Hence a philosopher is either eating or thinking or transitioning between these two states.

此外，哲学家们必须都是身体非常亲密、高度灵巧且接种了 COVID-19 疫苗的朋友，因为他们共用筷子吃饭。桌子上只有五根筷子，放在每个哲学家之间。当一位哲学家想要吃饭时，他们遵循先拿起左手筷子，然后拿起右手筷子的协议。一旦他们准备好再次思考，他们就会先归还右边的筷子，然后再归还左边的筷子。

In addition, the philosophers must all be physically very close, highly dexterous, and COVID-19 vaccinated friends, as they share chopsticks to eat. Only five chopsticks are on the table, placed between each philosopher. When one philosopher wishes to eat, they follow a protocol of picking up their left chopstick first, then their right chopstick. Once they are ready to think again, they first return the right chopstick, then the left.

图 4-4描绘了我们的哲学家，每个哲学家都由一个唯一的数字标识。由于每个哲学家要么同时吃饭要么思考，我们可以将每个哲学家建模为一条线程。

Figure 4-4 depicts our philosophers, each identified by a unique number. As each is either concurrently eating or thinking, we can model each philosopher as a thread.

图 4-4。哲学家就餐问题

代码如例 4-1所示。共享筷子由 Java 类的实例表示Object。由于任何时候只有一个对象可以持有一个对象的监视器锁，因此它们被用作临界区的进入条件，哲学家在临界区中获取他们需要吃的筷子。吃完后，将筷子放回桌子上，并释放每根筷子的锁，以便邻近的哲学家可以在准备好时就餐。

The code is shown in Example 4-1. The shared chopsticks are represented by instances of the Java Object class. As only one object can hold the monitor lock on an object at any time, they are used as entry conditions to the critical sections in which the philosophers acquire the chopsticks they need to eat. After eating, the chopsticks are returned to the table and the lock is released on each so that neighboring philosophers can eat whenever they are ready.

例 4-1。哲学家的线索

公共类 Philosopher 实现 Runnable {

  私有最终对象 leftChopStick;
  私有最终对象 rightChopStick;

  哲学家（对象 leftChopStick，对象 rightChopStick）{
    this.leftChopStick = leftChopStick;
    this.rightChopStick = rightChopStick;
  }
  私人无效LogEvent（字符串事件）抛出InterruptedException {
    System.out.println(Thread.currentThread()
                                  .getName() + " " + 事件);
    线程睡眠（1000）；
  }

  公共无效运行（）{
    尝试 {
      而（真）{
        LogEvent(": 深入思考");
        同步（左ChopStick）{
          LogEvent(": 拿起左筷子");
          同步（rightChopStick）{
            LogEvent(": 拿起右手筷子——吃饭");
            LogEvent(": 放下右手筷子");
          }
          LogEvent(": 放下左筷子，吃太多了");
        }
      } // 结束同时
    } catch (InterruptedException e) {
       Thread.currentThread().interrupt();
  }
 }
}

public class Philosopher implements Runnable {

  private final Object leftChopStick;
  private final Object rightChopStick;

  Philosopher(Object leftChopStick, Object rightChopStick) {
    this.leftChopStick = leftChopStick;
    this.rightChopStick = rightChopStick;
  }
  private void LogEvent(String event) throws InterruptedException {
    System.out.println(Thread.currentThread()
                                  .getName() + " " + event);
    Thread.sleep(1000);
  }

  public void run() {
    try {
      while (true) {
        LogEvent(": Thinking deeply"); 
        synchronized (leftChopStick) {
          LogEvent( ": Picked up left chopstick");
          synchronized (rightChopStick) {
            LogEvent(": Picked up right chopstick – eating");
            LogEvent(": Put down right chopstick");
          }
          LogEvent(": Put down left chopstick. Ate too much");
        }
      } // end while
    } catch (InterruptedException e) {
       Thread.currentThread().interrupt();
  }
 }
}

为了让例 4-1中描述的哲学家栩栩如生，我们必须为每个哲学家实例化一个线程，并让每个哲学家都能访问其相邻的筷子。这是通过例 4-2中的线程构造函数调用来完成的。在循环中，我们创建了五个哲学家，并将它们作为独立的线程启动，其中每根筷子可由两个线程访问，一个作为左筷子，一个作为右筷子。for

To bring the philosophers described in Example 4-1 to life, we must instantiate a thread for each and give each philosopher access to their neighboring chopsticks. This is done through the thread constructor call at in Example 4-2. In the for loop we create five philosophers and start these as independent threads, where each chopstick is accessible to two threads, one as a left chopstick, and one as a right.

例 4-2。哲学家就餐——僵局版本

私有最终静态 int NUMCHOPSTICKS = 5 ；
私有最终静态 int NUMPHILOSOPHERS = 5;
公共静态无效主（字符串[] args）抛出异常{
 
  最终哲学家[] ph =新哲学家[NUMPHILOSOPHERS];
  Object[] ChopSticks = new Object[NUMCHOPSTICKS];
 
  for (int i = 0; i < NUMCHOPSTICKS; i++) {
    ChopSticks[i] = new Object();
  }
 
  for (int i = 0; i < 数字; i++) {
    对象 leftChopStick = ChopSticks[i];
    对象rightChopStick=chopSticks[(i+1)%chopSticks.length];
    
    ph[i] = new Philosopher(leftChopStick, rightChopStick);  
          
    Thread th = new Thread(ph[i], "哲学家" + i);
    th.start();
  }
}

private final static int NUMCHOPSTICKS = 5 ;
private final static int NUMPHILOSOPHERS = 5; 
public static void main(String[] args) throws Exception {
 
  final Philosopher[] ph = new Philosopher[NUMPHILOSOPHERS];
  Object[] chopSticks = new Object[NUMCHOPSTICKS];
 
  for (int i = 0; i < NUMCHOPSTICKS; i++) {
    chopSticks[i] = new Object();
  }
 
  for (int i = 0; i < NUMPHILOSOPHERS; i++) {
    Object leftChopStick = chopSticks[i];
    Object rightChopStick = chopSticks[(i + 1) % chopSticks.length];
    
    ph[i] = new Philosopher(leftChopStick, rightChopStick);  
          
    Thread th = new Thread(ph[i], "Philosopher " + i);
    th.start();
  }
}

在我第一次尝试时运行此代码会产生以下输出。如果运行代码，您几乎肯定会看到不同的输出，但最终结果将是相同的：

Running this code produces the following output on my first attempt. If you run the code you will almost certainly see different outputs, but the final outcome will be the same:

哲学家3：深入思考
哲学家4：深入思考
哲学家0：深入思考
哲学家1：深入思考
哲学家2：深入思考
哲学家3：拿起左筷子
哲学家0：拿起左边的筷子
哲学家2：拿起左边的筷子
哲学家4：拿起左筷子
哲学家1：拿起左边的筷子

Philosopher 3 : Thinking deeply
Philosopher 4 : Thinking deeply
Philosopher 0 : Thinking deeply
Philosopher 1 : Thinking deeply
Philosopher 2 : Thinking deeply
Philosopher 3 : Picked up left chopstick
Philosopher 0 : Picked up left chopstick
Philosopher 2 : Picked up left chopstick
Philosopher 4 : Picked up left chopstick
Philosopher 1 : Picked up left chopstick

十行输出，然后……什么也没有！我们陷入了僵局。这是一个经典的循环等待死锁。想象一下以下场景：

Ten lines of output, then…nothing! We have a deadlock. This is a classic circular waiting deadlock. Imagine the following scenario:

• 每个哲学家都沉迷于长时间的思考。

• Each philosopher indulges in a long thinking session.

• 与此同时，他们都觉得自己饿了，于是伸手去拿左边的筷子。

• Simultaneously, they all decide they are hungry and reach for their left chopstick.

• 没有哲学家可以吃饭（继续），就像没有人可以拿起右手的筷子一样。

• No philosopher can eat (proceed) as none can pick up their right chopstick.

在这种情况下，真正的哲学家会想出一些办法，放下一两根筷子，直到他们的一个或多个同事可以吃饭。有时我们可以通过在阻塞操作上使用超时来在我们的软件中做到这一点。当超时到期时，线程释放临界区并重试，从而允许其他被阻塞的线程有机会继续执行。但这并不是最佳选择，因为阻塞线程会损害性能，并且设置超时值是一门不精确的科学。

Real philosophers in this situation would figure out some way to proceed by putting down a chopstick or two until one or more of their colleagues can eat. We can sometimes do this in our software by using timeouts on blocking operations. When the timeout expires, a thread releases the critical section and retries, allowing other blocked threads a chance to proceed. This is not optimal though, as blocked threads hurt performance and setting timeout values is an inexact science.

因此，最好设计一个无死锁的解决方案。这意味着一个或多个线程始终能够取得进展。对于循环等待死锁，这可以通过在共享资源上施加资源分配协议来实现，这样线程就不会总是以相同的顺序请求资源。

It is much better, therefore, to design a solution to be deadlock-free. This means that one or more threads will always be able to make progress. With circular wait deadlocks, this can be achieved by imposing a resource allocation protocol on the shared resources, so that threads will not always request resources in the same order.

在哲学家就餐问题中，我们可以通过确保其中一位哲学家首先拿起他正确的筷子来做到这一点。假设我们指示哲学家 4 这样做。这导致可能的操作顺序如下：

In the dining philosophers problem, we can do this by making sure one of our philosophers picks up their right chopstick first. Let’s assume we instruct Philosopher 4 to do this. This leads to a possible sequence of operations such as below:

• 哲学家 0 拿起左筷子 ( chopStick[0])，然后拿起右筷子 ( chopStick[1])

• Philosopher 0 picks up left chopstick (chopStick[0]) then right (chopStick[1])

• 哲学家 1 拿起左筷子 ( chopStick[1])，然后拿起右筷子 ( chopStick[2])

• Philosopher 1 picks up left chopstick (chopStick[1]) then right (chopStick[2])

• 哲学家 2 拿起左筷子 ( chopStick[2])，然后拿起右筷子 ( chopStick[3])

• Philosopher 2 picks up left chopstick (chopStick[2]) then right (chopStick[3])

• 哲学家 3 拿起左筷子 ( chopStick[3])，然后拿起右筷子 ( chopStick[4])

• Philosopher 3 picks up left chopstick (chopStick[3]) then right (chopStick[4])

• 哲学家 4 拿起右手筷子 ( chopStick[0])，然后拿起左筷子 ( chopStick[4])

• Philosopher 4 picks up right chopstick (chopStick[0]) then left (chopStick[4])

在此示例中，哲学家 4 必须阻止，因为哲学家 0 已经获得了对 的访问权限chopstick[0]。当哲学家 4 被阻止时，哲学家 3 就可以确保访问chopstick[4]并继续满足他们的胃口。

In this example, Philosopher 4 must block, as Philosopher 0 already has acquired access to chopstick[0]. With Philosopher 4 blocked, Philosopher 3 is assured access to chopstick[4] and can then proceed to satisfy their appetite.

哲学家就餐解决方案的修复如示例 4-3所示。

The fix for the dining philosophers solution is shown in Example 4-3.

例 4-3。解决哲学家就餐僵局

if (i == 数字哲学家 - 1) {
  // 最后一位哲学家先拿起右边的筷子
  ph[i] = new Philosopher(rightChopStick, leftChopStick);
} 别的 {
  // 其他人先拿起左边的筷子
  ph[i] = new Philosopher(leftChopStick, rightChopStick);
}

if (i == NUMPHILOSOPHERS - 1) {
  // The last philosopher picks up the right chopstick first
  ph[i] = new Philosopher(rightChopStick, leftChopStick); 
} else {
  // all others pick up the left chopstick first 
  ph[i] = new Philosopher(leftChopStick, rightChopStick);
}

更正式地说，我们对共享资源的获取施加命令，例如：

More formally we are imposing an ordering on the acquisition of shared resources, such that:

chopStick[0]<<<< chopStick[1]_ chopStick[2]_ chopStick[3]_chopStick[4]

chopStick[0] < chopStick[1] < chopStick[2] < chopStick[3] < chopStick[4]

这意味着每个线程将始终尝试获取chopstick[0]before chopstick[1]、chopstick[1]beforechopstick[2]等。对于 Philosopher 4 来说，这意味着他们将尝试获取chopstick[0]before chopstick[4]，从而打破循环等待死锁的可能性。

This means each thread will always attempt to acquire chopstick[0] before chopstick[1], and chopstick[1] before chopstick[2], and so on. For Philosopher 4, this means they will attempt to acquire chopstick[0] before chopstick[4], thus breaking the potential for a circular wait deadlock.

死锁是一个复杂的主题，本节仅触及表面。您会在许多分布式系统中看到死锁。例如，用户请求获取学生数据库表中某些数据的锁定，然后必须更新班级表中的行以反映学生出勤情况。同时另一个用户请求获取Classes表上的锁，接下来必须更新Students表中的一些信息。如果这些请求交织在一起，使得每个请求请求以重叠的方式获取锁，我们遇到了死锁。

Deadlocks are a complicated topic and this section has just scratched the surface. You’ll see deadlocks in many distributed systems. For example, a user request acquires a lock on some data in a Students database table, and must then update rows in the Classes table to reflect student attendance. Simultaneously another user request acquires locks on the Classes table, and next must update some information in the Students table. If these requests interleave such that each request acquires locks in an overlapping fashion, we have a deadlock.

线程状态

Thread States

多线程系统有一个系统调度程序来决定何时运行哪些线程。在 Java 中，调度程序被称为抢占式、基于优先级的调度程序。简而言之，这意味着它选择执行希望运行的最高优先级线程。

Multithreaded systems have a system scheduler that decides which threads to run when. In Java, the scheduler is known as a preemptive, priority-based scheduler. In short, this means it chooses to execute the highest priority thread which wishes to run.

每个线程都有一个优先级（默认为 5，范围为 0 到 10）。线程从其父线程继承其优先级。较高优先级线程比较低优先级线程更频繁地调度，但在大多数应用程序中，将所有线程设置为默认优先级就足够了。

Every thread has a priority (by default 5, range 0 to 10). A thread inherits its priority from its parent thread. Higher priority threads get scheduled more frequently than lower priority threads, but in most applications having all threads as the default priority suffices.

调度程序根据线程的行为在四种不同的状态之间循环线程。这些都是：

The scheduler cycles threads through four distinct states, based on their behavior. These are:

已创建

Created

线程对象已创建但其start()方法尚未被调用。一旦start()被调用，线程就进入可运行状态。

A thread object has been created but its start() method has not been invoked. Once start() is invoked, the thread enters the runnable state.

可运行

Runnable

线程能够运行。调度程序将选择以先进先出 (FIFO) 方式执行的线程 - 一个线程可以随时分配给节点中的每个核心。然后，线程执行，直到它们阻塞（例如，在synchronized语句上）、执行yield()、suspend()或sleep()语句、run()方法终止或被调度程序抢占。当较高优先级的线程变得可运行时，或者当系统特定的时间段（称为时间片）到期时，就会发生抢占。基于时间切片的抢占使调度程序能够确保所有线程最终都有机会执行——任何需要执行的线程都不会占用 CPU。

A thread is able to run. The scheduler will choose which thread(s) to execute in a first-in, first-out (FIFO) manner—one thread can be allocated at any time to each core in the node. Threads then execute until they block (e.g., on a synchronized statement), execute a yield(), suspend(), or sleep() statement, the run() method terminates, or they are preempted by the scheduler. Preemption occurs when a higher priority thread becomes runnable, or when a system-specific time period, known as a time slice, expires. Preemption based on time slicing allows the scheduler to ensure that all threads eventually get a chance to execute—no execution-hungry threads can hog the CPU.

被阻止

Blocked

如果线程正在等待，则该线程将被阻塞对于锁，通知事件发生（例如，睡眠定时器到期，resume()方法执行），或者正在等待网络或磁盘请求完成。当阻塞线程正在等待的特定事件发生时，它会返回到可运行状态。

A thread is blocked if it is waiting for a lock, a notification event to occur (e.g., sleep timer to expire, resume() method executed), or is waiting for a network or disk request to complete. When the specific event a blocked thread is waiting for occurs, it moves back to the runnable state.

终止

Terminated

线程的run()方法有已完成或已调用该stop()方法。该线程将不再被调度。

A thread’s run() method has completed or it has called the stop() method. The thread will no longer be scheduled.

该方案的示例如图4-5所示。调度程序有效地将每个线程优先级的 FIFO 队列维护在可运行状态。高优先级线程通常用于响应事件（例如，紧急计时器）并在短时间内执行。低优先级线程用于后台持续的任务，例如通过重新计算校验和来检查磁盘上的文件是否损坏。后台线程基本上会耗尽空闲的 CPU 周期。

An illustration of this scheme is in Figure 4-5. The scheduler effectively maintains FIFO queue in the runnable state for each thread priority. High-priority threads are typically used to respond to events (e.g., an emergency timer) and execute for a short period of time. Low-priority threads are used for background, ongoing tasks like checking for corruption of files on disk through recalculating checksums. Background threads basically use up idle CPU cycles.

图 4-5。线程状态和转换

线程协调

Thread Coordination

有很多问题需要线程具有不同的角色来协调他们的活动。想象一下一组线程，每个线程接受来自用户的文档，对文档进行一些处理（例如，生成 PDF），然后将处理后的文档发送到共享打印机池。每台打印机一次只能打印一个文档，因此它们从共享打印队列中读取，按照文档到达的顺序抓取并打印文档。

There are many problems that require threads with different roles to coordinate their activities. Imagine a collection of threads that each accept documents from users, do some processing on the documents (e.g., generate a PDF), and then send the processed document to a shared printer pool. Each printer can only print one document at a time, so they read from a shared print queue, grabbing and printing documents in the order they arrive.

这个打印问题是经典的生产者-消费者问题的一个例证。生产者产生并通过共享 FIFO 缓冲区向消费者发送消息。消费者检索这些消息，处理它们，然后从缓冲区请求更多工作。图 4-6显示了该问题的简单说明。这有点像一家365天24小时营业的自助餐厅——厨房不停地生产，服务员收集食物并放入自助餐中，饥饿的食客可以自助。永远。

This printing problem is an illustration of the classic producer-consumer problem. Producers generate and send messages via a shared FIFO buffer to consumers. Consumers retrieve these messages, process them, and then ask for more work from the buffer. A simple illustration of this problem is shown in Figure 4-6. It’s a bit like a 24-hour, 365-day buffet restaurant—the kitchen keeps producing, the waitstaff collect the food and put it in the buffet, and hungry diners help themselves. Forever.

图 4-6。生产者-消费者问题

与几乎所有实际资源一样，缓冲区的容量是有限的。生产者生成新的项目，但如果缓冲区已满，他们必须等到某些项目被消耗后才能将新项目添加到缓冲区。类似地，如果消费者的消费速度比生产者的生产速度快，那么如果缓冲区中没有项目，他们必须等待，并在新项目到达时以某种方式收到警报。

Like virtually all real resources, the buffer has a limited capacity. Producers generate new items, but if the buffer is full, they must wait until some item(s) have been consumed before they can add the new item to the buffer. Similarly, if the consumers are consuming faster than the producers are producing, then they must wait if there are no items in the buffer, and somehow get alerted when new items arrive.

生产者等待缓冲区中的空间或消费者等待项目的一种方法是不断重试操作。生产者可以休眠一秒钟，然后重试 put 操作，直到成功。消费者也可以这样做。

One way for a producer to wait for space in the buffer, or a consumer to wait for an item, is to keep retrying an operation. A producer could sleep for a second, and then retry the put operation until it succeeds. A consumer could do likewise.

这个解决方案称为 轮询，或忙等待。它工作得很好，但正如第二个名字所暗示的那样，每个生产者和消费者每次重试和失败时都在使用资源（CPU、内存，也许是网络？）。如果这不是问题，那么很酷，但在可扩展系统中，我们始终致力于优化资源使用，而轮询可能会造成浪费。

This solution is called polling, or busy waiting. It works fine, but as the second name implies, each producer and consumer are using resources (CPU, memory, maybe network?) each time they retry and fail. If this is not a concern, then cool, but in scalable systems we are always aiming to optimize resource usage, and polling can be wasteful.

更好的解决方案是生产者和消费者阻塞，直到他们想要的操作（分别放置或获取）能够成功。阻塞线程不消耗资源，因此提供了一种有效的解决方案。为了实现这一点，线程编程模型提供了阻塞操作，使线程能够在事件发生时向其他线程发出信号。对于生产者-消费者问题，基本方案如下：

A better solution is for producers and consumers to block until their desired operation, put or get respectively, can succeed. Blocked threads consume no resources and hence provide an efficient solution. To facilitate this, thread programming models provide blocking operations that enable threads to signal to other threads when an event occurs. With the producer-consumer problem, the basic scheme is as follows:

• 当生产者将一个项目添加到缓冲区时，它会向任何阻塞的消费者发送一个信号，通知他们缓冲区中有一个项目。

• When a producer adds an item to the buffer, it sends a signal to any blocked consumers to notify them that there is an item in the buffer.

• 当消费者从缓冲区检索项目时，它会向任何被阻止的生产者发送信号，通知他们缓冲区中有新项目的容量。

• When a consumer retrieves an item from the buffer, it sends a signal to any blocked producers to notify them there is capacity in the buffer for new items.

在Java中，有两个基本的原语，即wait()和notify()，可用于实现该信令方案。简而言之，它们的工作原理如下：

In Java, there are two basic primitives, namely wait() and notify(), that can be used to implement this signaling scheme. Briefly, they work like this:

• 如果线程wait()需要保持的某些条件不成立，则线程可以在同步块内调用。例如，一个线程可能尝试从缓冲区检索消息，但如果缓冲区没有要检索的消息，则它会调用wait()并阻塞，直到另一个线程添加消息、将条件设置为true并调用notify()同一对象。

• A thread may call wait() within a synchronized block if some condition it requires to hold is not true. For example, a thread may attempt to retrieve a message from a buffer, but if the buffer has no messages to retrieve, it calls wait() and blocks until another thread adds a message, sets the condition to true, and calls notify() on the same object.

• notify()唤醒调用该对象的线程wait()。

• notify() wakes up a thread that has called wait() on the object.

这些 Java 原语用于实施受保护的块。受保护的块使用条件作为保护，该条件在线程恢复执行之前必须保持。以下代码片段显示了如何使用保护条件empty来阻止尝试从空缓冲区检索消息的线程：

These Java primitives are used to implement guarded blocks. Guarded blocks use a condition as a guard that must hold before a thread resumes the execution. The following code snippet shows how the guard condition, empty, is used to block a thread that is attempting to retrieve a message from an empty buffer:

而（空）{
  尝试 {
    System.out.println("等待消息");
    等待（）;
  } catch (InterruptedException e) {}
}

while (empty) {
  try {
    System.out.println("Waiting for a message");
    wait();
  } catch (InterruptedException e) {}
}

当另一个线程向缓冲区添加消息时，它执行notify()如下：

When another thread adds a message to the buffer, it executes notify() as follows:

// 存储消息。
this.message = 消息；
空=假；
// 通知消费者消息可用
通知（）;

// Store message.
this.message = message;
empty = false;
// Notify consumer that message is available
notify();

Git 存储库一书中的代码示例给出了此示例的完整实现。wait()和方法有许多变体notify()，但这些超出了我在本概述中可以涵盖的范围。幸运的是，Java 为我们提供了线程安全的抽象，可以隐藏代码中的这种复杂性。

The full implementation of this example is given in the code examples in the book Git repository. There are a number of variations of the wait() and notify() methods, but these go beyond the scope of what I can cover in this overview. And luckily, Java provides us with thread-safe abstractions that hide this complexity from your code.

与生产者-消费者问题相关的一个例子BlockingQueue是java.util.concurrent.BlockingQueue. 一个BlockingQueue实现提供了一个线程安全的对象，可以用作生产者-消费者场景中的缓冲区。该接口有 5 种不同的实现BlockingQueue。我将使用其中之一LinkedBlockingQueue来实现生产者-消费者。示例 4-4中显示了这一点。

An example that is pertinent to the producer-consumer problem is the BlockingQueue interface in java.util.concurrent.BlockingQueue. A BlockingQueue implementation provides a thread-safe object that can be used as the buffer in a producer-consumer scenario. There are 5 different implementations of the BlockingQueue interface. I’ll use one of these, the LinkedBlockingQueue, to implement the producer-consumer. This is shown in Example 4-4.

生产者消费者类 {
   公共静态无效主（字符串[]参数）
     BlockingQueue 缓冲区 = new LinkedBlockingQueue();
     生产者 p = 新生产者（缓冲区）；
     消费者 c = 新消费者（缓冲区）；
     新线程(p).start();
     新线程(c).start();
   }
 }

类 Producer 实现 Runnable {
   私有布尔活动= true；
   私有最终 BlockingQueue 缓冲区；
   公共生产者（BlockingQueue q）{缓冲区= q; }
   公共无效运行（）{
     
     尝试 {
       while (active) { buffer.put(product()); }
     } catch (InterruptedException ex) { // 处理异常}
   }
   Object Produce() { // 省略细节，设置 active=false }
 }

 类 Consumer 实现 Runnable {
   私有布尔活动= true；  
   私有最终 BlockingQueue 缓冲区；
   公共消费者（BlockingQueue q）{缓冲区= q; }
   公共无效运行（）{
     
     尝试 {
       while (active) { 消耗(buffer.take()); }
     } catch (InterruptedException ex) { // 处理异常 }
   }
   void Consumer(Object x) { // 省略细节，设置 active=false }
 }

class ProducerConsumer {
   public static void main(String[] args)
     BlockingQueue buffer = new LinkedBlockingQueue();
     Producer p = new Producer(buffer);
     Consumer c = new Consumer(buffer);
     new Thread(p).start();
     new Thread(c).start();
   }
 }

class Producer implements Runnable {
   private boolean active = true;
   private final BlockingQueue buffer;
   public Producer(BlockingQueue q) { buffer = q; }
   public void run() {
     
     try {
       while (active) { buffer.put(produce()); }
     } catch (InterruptedException ex) { // handle exception}
   }
   Object produce() { // details omitted, sets active=false }
 }

 class Consumer implements Runnable {
   private boolean active = true;  
   private final BlockingQueue buffer;
   public Consumer(BlockingQueue q) { buffer = q; }
   public void run() {
     
     try {
       while (active) { consume(buffer.take()); }
     } catch (InterruptedException ex) { // handle exception }
   }
   void consume(Object x) {  // details omitted, sets active=false }
 }

该解决方案使程序员无需关心协调对共享缓冲区的访问的实现，并且大大简化了代码。

This solution absolves the programmer from being concerned with the implementation of coordinating access to the shared buffer, and greatly simplifies the code.

该java.util.concurrent 包是构建多线程 Java 解决方案的宝库。在以下各节中，我将简要强调其中的一些内容强大且极其有用的功能。

The java.util.concurrent package is a treasure trove for building multithreaded Java solutions. In the following sections, I will briefly highlight a few of these powerful and extremely useful capabilities.

线程池

Thread Pools

许多多线程系统需要创建并管理执行类似任务的线程集合。例如，在生产者-消费者问题中，我们可以拥有一组生产者线程和一组消费者线程，所有线程都同时添加和删除项目，并协调对共享缓冲区的访问。

Many multithreaded systems need to create and manage a collection of threads that perform similar tasks. For example, in the producer-consumer problem, we can have a collection of producer threads and a collection of consumer threads, all simultaneously adding and removing items, with coordinated access to the shared buffer.

这些集合称为线程池。线程池由多个工作线程组成，这些线程通常执行类似的目的并作为一个集合进行管理。我们可以创建一个生产者线程池，所有线程都等待一个项目进行处理，将最终产品写入缓冲区，然后等待接受另一个要处理的项目。当我们停止生产物品时，可以以安全的方式关闭池，因此不会因意外异常而丢失部分处理的物品。

These collections are known as thread pools. Thread pools comprise several worker threads, which typically perform a similar purpose and are managed as a collection. We could create a pool of producer threads which all wait for an item to process, write the final product to the buffer, and then wait to accept another item to process. When we stop producing items, the pool can be shut down in a safe manner, so no partially processed items are lost through an unanticipated exception.

在java.util.concurrent包中，线程接口支持池ExecutorService。这通过一组方法扩展了基本Executor接口来管理和终止池中的线程。示例4-5和4-6中显示了一个使用固定大小线程池的简单生产者-消费者示例。示例 4-5Producer中的类向缓冲区发送一条消息，然后终止。它只是从缓冲区中获取消息，直到收到空字符串为止，然后终止。RunnableConsumer

In the java.util.concurrent package, thread pools are supported by the ExecutorService interface. This extends the base Executor interface with a set of methods to manage and terminate threads in the pool. A simple producer-consumer example using a fixed size thread pool is shown in Examples 4-5 and 4-6. The Producer class in Example 4-5 is a Runnable that sends a single message to the buffer and then terminates. The Consumer simply takes messages from the buffer until an empty string is received, upon which it terminates.

类 Producer 实现 Runnable {
  
  私有最终 BlockingQueue 缓冲区；

  公共生产者（BlockingQueue q）{缓冲区= q; }

  @覆盖
  公共无效运行（）{
     
  尝试 {
    睡眠（1000）；
    buffer.put("你好世界");
              
  } catch (InterruptedException ex) {
    // 处理异常
  }
 }
}

类 Consumer 实现 Runnable {
  私有最终 BlockingQueue 缓冲区；

  公共消费者（BlockingQueue q）{缓冲区= q; }

  @覆盖
   公共无效运行（）{
      布尔活动 = true;
      同时（活动）{
          尝试 {
             String s = (String) buffer.take();
             System.out.println(s);
             if (s.equals("")) active = false;
          } catch (InterruptedException ex) {
              / 处理异常
          }
      } /
      System.out.println("消费者终止");
    }
 }

class Producer implements Runnable {
  
  private final BlockingQueue buffer;

  public Producer(BlockingQueue q) { buffer = q; }

  @Override
  public void run() {
     
  try {
    sleep(1000);
    buffer.put("hello world");
              
  } catch (InterruptedException ex) {
    // handle exception
  }
 } 
}

class Consumer implements Runnable {
  private final BlockingQueue buffer;

  public Consumer(BlockingQueue q) { buffer = q; }

  @Override
   public void run() {
      boolean active = true; 
      while (active) {
          try {
             String  s = (String) buffer.take();
             System.out.println(s);
             if (s.equals("")) active = false;
          } catch (InterruptedException ex) {
              / handle exception
          }
      } /
      System.out.println("Consumer terminating");
    }
 }

在示例 4-6中，我们创建一个消费者来从缓冲区获取消息。然后我们创建一个大小为 5 的固定大小线程池来管理我们的生产者。这会导致 JVM 预分配 5 个线程，这些线程可用于执行Runnable池中执行的任何对象。

In Example 4-6, we create a single consumer to take messages from the buffer. We then create a fixed size thread pool of size 5 to manage our producers. This causes the JVM to preallocate five threads that can be used to execute any Runnable objects that are executed by the pool.

然后，在for()循环中，我们使用ExecutorService来运行 20 个生产者。因为只有 5 个线程线程池中最多只能同时执行 5 个生产者。所有其他的都被放置在由线程池管理的等待队列中。当生产者终止时，Runnable等待队列中的下一个生产者将使用池中的任何可用线程执行。

In the for() loop, we then use the ExecutorService to run 20 producers. As there are only 5 threads available in the thread pool, only a maximum of 5 producers will be executed simultaneously. All others are placed in a wait queue which is managed by the thread pool. When a producer terminates, the next Runnable in the wait queue is executed using any available thread in the pool.

一旦我们请求线程池执行所有生产者，我们就调用shutdown()池上的方法。这告诉它ExecutorService不再接受任何要运行的任务。接下来我们调用该awaitTermination()方法，该方法会阻塞调用线程，直到线程池管理的所有线程都空闲并且等待队列中没有更多工作在等待。一旦awaitTermination()返回，我们知道所有消息都已发送到缓冲区，因此将一个空字符串发送到缓冲区，该缓冲区将充当消费者的终止值。

Once we have requested all the producers to be executed by the thread pool, we call the shutdown() method on the pool. This tells the ExecutorService not to accept any more tasks to run. We next call the awaitTermination() method, which blocks the calling thread until all the threads managed by the thread pool are idle and no more work is waiting in the wait queue. Once awaitTermination() returns, we know all messages have been sent to the buffer, and hence send an empty string to the buffer which will act as a termination value for the consumer.

public static void main(String[] args) 抛出 InterruptedException
  {
    BlockingQueue 缓冲区 = new LinkedBlockingQueue();
    
    //启动单个消费者
    (new Thread(new Consumer(buffer))).start();

    ExecutorService ProducerPool = Executors.newFixedThreadPool(5);
    for (int i = 0; i < 20; i++)
      {
        生产者生产者=新生产者（缓冲区）；
        System.out.println("生产者已创建");
        生产者池.execute(生产者);
      }

      生产者池.shutdown();
      ProducerPool.awaitTermination(10, TimeUnit.SECONDS);
        
      //向消费者发送终止消息
      缓冲区.put("");        
    }

public static void main(String[] args) throws InterruptedException 
  {
    BlockingQueue buffer = new LinkedBlockingQueue();
    
    //start a single consumer 
    (new Thread(new Consumer(buffer))).start();

    ExecutorService producerPool = Executors.newFixedThreadPool(5);
    for (int i = 0; i < 20; i++) 
      {
        Producer producer = new Producer(buffer) ;
        System.out.println("Producer created" );
        producerPool.execute(producer);
      }

      producerPool.shutdown();
      producerPool.awaitTermination(10, TimeUnit.SECONDS);
        
      //send termination message to consumer 
      buffer.put("");        
    }

与本章中的大多数主题一样，框架中有许多更复杂的功能Executor可用于创建多线程程序。此描述仅涵盖了基础知识。线程池很重要，因为它们使我们的系统能够合理地使用线程资源。每个线程都会消耗内存；例如，线程的堆栈大小通常约为 1 MB。此外，当我们切换执行上下文来运行新线程时，这会消耗 CPU 周期。如果我们的系统以无纪律的方式创建线程，我们最终将耗尽内存并且系统将崩溃。线程池允许我们控制创建的线程数量并有效地利用它们。

As with most topics in this chapter, there are many more sophisticated features in the Executor framework that can be used to create multithreaded programs. This description has just covered the basics. Thread pools are important as they enable our systems to rationalize the use of resources for threads. Every thread consumes memory; for example, the stack size for a thread is typically around 1 MB. Also, when we switch execution context to run a new thread, this consumes CPU cycles. If our systems create threads in an undisciplined manner, we will eventually run out of memory and the system will crash. Thread pools allow us to control the number of threads we create and utilize them efficiently.

我将在本书的其余部分讨论线程池，因为它们是高效和可扩展管理的关键概念。服务器必须满足不断增加的请求负载。

I’ll discuss thread pools throughout the remainder of this book, as they are a key concept for efficient and scalable management of the ever-increasing request loads that servers must satisfy.

屏障同步

Barrier Synchronization

我有一个高中朋友，他的家人在晚餐时间，在全家人都坐在餐桌旁之前，不允许任何人开始吃饭。我认为这很奇怪，但多年后，它可以作为屏障同步概念的一个很好的类比。当所有家庭成员都到达餐桌后才开始吃饭。

I had a high school friend whose family, at dinnertime, would not allow anyone to start eating until the whole family was seated at the table. I thought this was weird, but many years later it serves as a good analogy for the concept known as barrier synchronization. Eating commenced only after all family members arrived at the table.

多线程系统通常需要遵循这样的行为模式。想象一个多线程图像处理系统。图像到达后，图像的不同部分被传递到每个线程以执行一些转换 - 想想 Instagram 的类固醇过滤器。仅当所有线程都完成时，图像才会被完全处理。在软件系统中，我们使用一种称为屏障同步的机制来实现这种线程协调方式。

Multithreaded systems often need to follow such a pattern of behavior. Imagine a multithreaded image-processing system. An image arrives and a distinct segment of the image is passed to each thread to perform some transformation upon—think Instagram filters on steroids. The image is only fully processed when all threads have completed. In software systems, we use a mechanism called barrier synchronization to achieve this style of thread coordination.

总体方案如图4-7所示。在此示例中，main()线程创建了四个新线程，并且所有线程都独立执行，直到到达屏障定义的执行点。当每个线程到达时，它就会阻塞。当所有线程都到达这一点时，屏障就会被释放，每个线程都可以继续其处理。

The general scheme is shown in Figure 4-7. In this example, the main() thread creates four new threads and all proceed independently until they reach the point of execution defined by the barrier. As each thread arrives, it blocks. When all threads have arrived at this point, the barrier is released, and each thread can continue with its processing.

图 4-7。栅栏同步

Java 提供了三个原语屏障同步。我将在这里展示这三个之一是如何CountDownLatch工作的。基本概念适用于其他屏障同步原语。

Java provides three primitives for barrier synchronization. I’ll show here how one of the three, CountDownLatch, works. The basic concepts apply to other barrier synchronization primitives.

创建 时CountDownLatch，您将一个值传递给其构造函数，该值表示必须在屏障处阻塞的线程数，然后才允许它们继续运行。这在管理系统屏障点的线程中调用——在图 4-7中，这将是main()：

When you create a CountDownLatch, you pass a value to its constructor that represents the number of threads that must block at the barrier before they are all allowed to continue. This is called in the thread which is managing the barrier points for the system—in Figure 4-7 this would be main():

CountDownLatch nextPhaseSignal = new CountDownLatch(numThreads);

CountDownLatch  nextPhaseSignal = new CountDownLatch(numThreads);

接下来，您创建工作线程将执行一些操作，然后在障碍物处阻塞，直到全部完成。为此，您需要向每个线程传递一个引用CountDownLatch：

Next, you create the worker threads that will perform some actions and then block at the barrier until they all complete. To do this, you need to pass each thread a reference to CountDownLatch:

for (int i = 0; i < numThreads; i++) {
            线程工作者 = new Thread(new WorkerThread(nextPhaseSignal));
            工作人员.start();
        }

for (int i = 0; i < numThreads; i++) {
            Thread worker = new Thread(new WorkerThread(nextPhaseSignal));
            worker.start();
        }

启动工作线程后，main()线程将调用该.await()方法进行阻塞，直到工作线程触发闩锁：

After launching the worker threads, the main() thread will call the .await() method to block until the latch is triggered by the worker threads:

nextPhaseSignal.await();

nextPhaseSignal.await();

每个工作线程将完成其任务，并在退出之前调用.countDown()闩锁上的方法。这会减少锁存值。当最后一个线程调用.countDown()并且锁存器值变为零时，所有调用锁存器的线程都会从阻塞.await()状态转换为可运行状态。在此阶段，我们确信所有工作人员都已完成分配的任务：

Each worker thread will complete its task and, before exiting, call the .countDown() method on the latch. This decrements the latch value. When the last thread calls .countDown() and the latch value becomes zero, all threads that have called .await() on the latch transition from the blocked to the runnable state. At this stage we are assured that all workers have completed their assigned task:

nextPhaseSignal.countDown();

nextPhaseSignal.countDown();

.countDown()当闩锁已被有效触发时，任何后续调用都将立即返回。注意.countDown()是非阻塞，对于线程在到达屏障后还有更多工作要做的应用程序来说，这是一个有用的属性。

Any subsequent calls to .countDown() will return immediately as the latch has been effectively triggered. Note .countDown() is nonblocking, which is a useful property for applications in which threads have more work to do after reaching the barrier.

此示例说明使用 aCountDownLatch来阻止单个线程，直到一组线程完成其工作。但是，如果将其值初始化为 1，则可以使用锁存器反转此用例。多个线程可以调用.await()并阻塞，直到另一个线程调用.countDown()来释放所有等待线程。此示例类似于一个简单的门，一个线程打开该门以允许其他线程的集合继续。

This example illustrates using a CountDownLatch to block a single thread until a collection of threads have completed their work. You can invert this use case with a latch, however, if you initialize its value to one. Multiple threads could call .await() and block until another thread calls .countDown() to release all waiting threads. This example is analogous to a simple gate, which one thread opens to allow a collection of others to continue.

CountDownLatch是一个简单的屏障同步器。它是一次性工具，因为初始化值无法重置。CyclicBarrierJava 中的和类提供了更复杂的功能Phaser。有了本节中屏障同步如何工作的知识，这些将是简单易懂。

CountDownLatch is a simple barrier synchronizer. It’s a single-use tool, as the initializer value cannot be reset. More sophisticated features are provided by the CyclicBarrier and Phaser classes in Java. Armed with the knowledge of how barrier synchronization works from this section, these will be straightforward to understand.

线程安全集合

Thread-Safe Collections

很多Java程序员一旦深入研究深入了解多线程程序的奥妙后，惊讶地发现java.util包中的集合并不是线程安全的。²为什么，我听到你问？幸运的是，答案很简单。这与性能有关。调用同步方法会产生开销。因此，为了实现单线程程序的更快执行，集合不是线程安全的。

Many Java programmers, once they delve into the wonders of multithreaded programs, are surprised to discover that the collections in the java.util package are not thread safe.² Why, I hear you ask? The answer, luckily, is simple. It has to do with performance. Calling synchronized methods incurs overheads. Hence, to attain faster execution for single-threaded programs, the collections are not thread safe.

如果您想跨多个线程共享ArrayList、Map、 或您最喜欢的数据结构java.util，则必须确保对该结构的修改位于关键部分。这种方法给集合的客户端带来了安全更新的负担，因此很容易出错——程序员可能会忘记在块中进行修改synchronized。

If you want to share an ArrayList, Map, or your favorite data structure from java.util across multiple threads, you must ensure modifications to the structure are placed in critical sections. This approach places the burden on the client of the collection to safely make updates, and hence is error prone—a programmer might forget to make modifications in a synchronized block.

在多线程代码中使用本质上线程安全的集合总是更安全。因此，Java 集合框架提供了一个工厂方法来创建java.util集合的线程安全版本。这是创建线程安全列表的示例：

It’s always safer to use inherently thread-safe collections in your multithreaded code. For this reason, the Java collections framework provides a factory method that creates a thread-safe version of java.util collections. Here’s an example of creating a thread-safe list:

List<String> list = Collections.synchronizedList(new ArrayList<>());

List<String> list = Collections.synchronizedList(new ArrayList<>());

这里真正发生的事情是你围绕基集合类创建一个包装器，该类具有synchronized方法。当然，它们以线程安全的方式将实际工作委托给原始类。您可以对java.util包中的任何集合使用此方法，一般形式为：

What is really happening here is that you are creating a wrapper around the base collection class, which has synchronized methods. These delegate the actual work to the original class, in a thread-safe manner of course. You can use this approach for any collection in the java.util package, and the general form is:

Collections.synchronized....（新集合<>()）

Collections.synchronized....(new collection<>())

其中“ ....”是List、Map、Set、 等。

where “....” is List, Map, Set, and so on.

当然，当使用同步包装器时，您会因获取监视器锁和从多个线程串行访问而付出性能损失。这意味着当单个线程进行修改时整个集合被锁定，极大地限制了并发性能（再次阿姆达尔定律）。因此，Java 5.0 包含了并发集合包，即java.util.concurrent. 它包含专门为高效多线程访问而设计的丰富的类集合。

Of course, when using the synchronized wrappers, you pay the performance penalty for acquiring the monitor lock and serializing access from multiple threads. This means the whole collection is locked while a single thread makes a modification, greatly limiting concurrent performance (Amdahl’s law again). For this reason, Java 5.0 included the concurrent collections package, namely java.util.concurrent. It contains a rich collection of classes specifically designed for efficient multithreaded access.

事实上，我们已经见过其中一个类—— LinkedBlockingQueue. 这使用了一种锁定机制，可以并行地将项目添加到队列中或从队列中删除。这种更细粒度的锁定机制利用java.util.concurrent.lock.Lock类而不是监视器锁定方法。这允许在同一个集合上使用多个锁，从而实现安全的并发访问。

In fact, we’ve already seen one of these classes—the LinkedBlockingQueue. This uses a locking mechanism that enables items to be added to and removed from the queue in parallel. This finer grain locking mechanism utilizes the java.util.concurrent.lock.Lock class rather than the monitor lock approach. This allows multiple locks to be utilized on the same collection, hence enabling safe concurrent access.

提供这种更细粒度锁定的另一个非常有用的集合是ConcurrentHashMap. 这提供了与非线程安全类似的方法HashMap，但允许基于concurrencyLevel可以传递给构造函数的值进行非阻塞读取和并发写入（默认值为 16）：

Another extremely useful collection that provides this finer-grain locking is the ConcurrentHashMap. This provides the similar methods as the non–thread safe HashMap, but allows nonblocking reads and concurrent writes based on a concurrencyLevel value you can pass to the constructor (the default value is 16):

ConcurrentHashMap（int 初始容量，浮点 loadFactor，
                     int 并发级别）

ConcurrentHashMap (int initialCapacity, float loadFactor, 
                     int concurrencyLevel)

在内部，哈希表是分为单独可锁定的段，通常称为分片。锁与每个分片相关联，而不是与整个集合相关联。这意味着可以同时对集合的不同分片中的哈希表条目进行更新，从而提高性能。

Internally, the hash table is divided into individually lockable segments, often known as shards. Locks are associated with each shard rather than the whole collection. This means updates can be made concurrently to hash table entries in different shards of the collection, increasing performance.

出于性能原因，检索操作是非阻塞的，这意味着它们可以与多个并发更新重叠。这意味着检索仅反映执行检索时最近完成的更新操作的结果。

Retrieval operations are nonblocking for performance reasons, meaning they can overlap with multiple concurrent updates. This means retrievals only reflect the results of the most recently completed update operations at the time the retrieval is executed.

出于类似的原因，迭代器因为 aConcurrentHashMap是所谓的弱一致。这意味着迭代器包含哈希映射的副本，该副本反映了迭代器创建时的状态。当迭代器正在使用时，可以添加新节点并从底层哈希图中删除现有节点。然而，这些状态变化并没有反映在迭代器中。

For similar reasons, iterators for a ConcurrentHashMap are what is known as weakly consistent. This means the iterator contains a copy of the hash map that reflects its state at the time the iterator is created. While the iterator is in use, new nodes may be added and existing nodes removed from the underlying hash map. However, these state changes are not reflected in the iterator.

如果您需要一个始终反映当前哈希图状态并同时由多个线程更新的迭代器，那么就会付出性能损失，而 a 则ConcurrentHashMap不会正确的方法。这是一个优先考虑性能而不是一致性的例子——一个经典的设计权衡。

If you need an iterator that always reflects the current hashmap state while being updated by multiple threads, then there are performance penalties to pay, and a ConcurrentHashMap is not the right approach. This is an example of favoring performance over consistency—a classic design trade-off.

总结和延伸阅读

Summary and Further Reading

我将在本书的其余部分中利用本章中介绍的主要概念。线程本质上是我们用来构建可扩展分布式系统的数据处理和数据库平台的组件。在许多情况下，您可能没有明确编写多线程代码。但是，您编写的代码将在多线程环境中调用，这意味着您需要注意线程安全。许多平台还通过配置参数公开其并发性，这意味着要调整系统的性能，您需要了解更改各种线程和线程池设置的影响。基本上，在可扩展的分布式系统的世界中，并发是不可避免的。

I’ll draw upon the major concepts introduced in this chapter throughout the remainder of this book. Threads are inherently components of the data processing and database platforms that we use to build scalable distributed systems. In many cases, you may not be writing explicitly multithreaded code. However, the code you write will be invoked in a multithreaded environment, which means you need to be aware of thread safety. Many platforms also expose their concurrency through configuration parameters, meaning that to tune the system’s performance, you need to understand the effects of changing the various threading and thread pool settings. Basically, there’s no escaping concurrency in the world of scalable distributed systems.

最后，值得一提的是，虽然并发编程原语因编程语言而异，但基本问题不会改变，并且需要仔细设计多线程代码以避免竞争条件和死锁。无论您是处理C/C++ 中的pthreads库，还是受 CSP 启发的经典 Go 并发模型，您需要避免的问题都是相同的。无论您使用什么语言，您从本章中获得的知识都将使您走上正确的道路。

Finally, it is worth mentioning that while concurrent programming primitives vary across programming languages, the foundational issues don’t change, and carefully designed multithreaded code to avoid race conditions and deadlocks is needed. Whether you grapple with the pthreads library in C/C++, or the classic CSP-inspired Go concurrency model, the problems you need to avoid are the same. The knowledge you have gained from this chapter will regardless set you on the right track, whatever language you are using.

本章仅简单介绍了并发性及其在 Java 中的支持。继续学习有关并发基本概念的最佳书籍是Brian Goetz 等人编写的经典Java Concurrency in Practice ( JCiP )。（艾迪生-韦斯利专业人士，2006 年）。如果您理解本书中的所有内容，您将编写出非常出色的并发代码。

This chapter has only brushed the surface of concurrency in general and its support in Java. The best book to continue learning more about the basic concepts of concurrency is the classic Java Concurrency in Practice (JCiP) by Brian Goetz et al. (Addison-Wesley Professional, 2006). If you understand everything in the book, you’ll be writing pretty great concurrent code.

自 Java 5 以来，Java 并发支持有了很大的发展。在 Java 12（或您阅读本文时的任何当前版本）的世界中，出现了 CompletableFutures、lambda 表达式和并行流等新功能。Java 8.0 中引入的函数式编程风格可以轻松创建并发解决方案，而无需直接创建和管理线程。有关 Java 8.0 功能的一个很好的知识来源是Javier Fernández González 所著的《Mastering Concurrency Programming with Java 8》 （Packt，2017 年）。

Java concurrency support has moved on considerably since Java 5. In the world of Java 12 (or whatever version is current when you read this), there are new features such as CompletableFutures, lambda expressions, and parallel streams. The functional programming style introduced in Java 8.0 makes it easy to create concurrent solutions without directly creating and managing threads. A good source of knowledge for Java 8.0 features is Mastering Concurrency Programming with Java 8 by Javier Fernández González (Packt, 2017).

其他优秀来源包括：

Other excellent sources include:

• Doug Lea，Java 并发编程：设计原则和模式，第二版。（艾迪生-韦斯利专业人士，1996 年）

• Doug Lea, Concurrent Programming in Java: Design Principles and Patterns, 2nd ed. (Addison-Wesley Professional, 1996)

• Raoul-Gabriel Urma、Mario Fusco 和 Alan Mycroft，现代 Java 实践：Lambda、流、函数式和响应式编程（Manning，2019 年）

• Raoul-Gabriel Urma, Mario Fusco, and Alan Mycroft, Modern Java in Action: Lambdas, Streams, Functional and Reactive Programming (Manning, 2019)

• Baeldung网站收集了有关 Java 并发性的全面学习文章，并作为本章哲学家就餐示例的基础。

• The Baeldung website has a comprehensive collection of articles for learning about Java concurrency and served as the basis for the dining philosophers example in this chapter.

服务设计

Service Design

在最简单的情况下，应用程序包含一个面向互联网的服务，该服务将数据保存到本地数据存储，如图5-1所示。客户端通过其发布的 API 与服务交互，该 API 可通过互联网访问。

In the simplest case, an application comprises one internet facing service that persists data to a local data store, as shown in Figure 5-1. Clients interact with the service through its published API, which is accessible across the internet.

图 5-1。简单的服务

让我们更详细地了解 API 和服务实现。

Let’s look at the API and service implementation in more detail.

获取/skico.com/skiers/12345

GET /skico.com/skiers/12345

{
    “用户名”：“Ian123”，
    “电子邮件”：“i.gorton@somewhere.com”
    “城市”：“西雅图”
}

{
    "username": "Ian123",
    "email": "i.gorton@somewhere.com"
    "city": "Seattle"
}

放置 /skico.com/skiers/12345
{
    “用户名”：“Ian123”，
    “电子邮件”：“i.gorton@somewhere.com”
    “城市”：“韦纳奇”
}

PUT  /skico.com/skiers/12345
{
    "username": "Ian123",
    "email": "i.gorton@somewhere.com"
    "city": "Wenatchee"
}

/skico.com/skiers/768934

/skico.com/skiers/768934

路径：
  /度假村：
    得到：
      标签：
        - 度假村
      摘要：获取数据库中的滑雪场列表
      操作 ID：getResorts
      回应：
        “200”：
          描述：运行成功
          内容：
            应用程序/json：
              架构：
                $ref: '#/components/schemas/ResortsList'
        ‘404’：
          描述：未找到度假村。除非我们破产，否则不太可能
          内容：
            应用程序/json：
              架构：
                $ref: '#/components/schemas/responseMsg

paths:
  /resorts:
    get:
      tags:
        - resorts
      summary: get a list of ski resorts in the database
      operationId: getResorts
      responses:
        '200':
          description: successful operation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ResortsList'
        '404':
          description: Resorts not found. Unlikely unless we go broke
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/responseMsg

app.get('/skiers/:skierID', function (req, res) {
  // 处理GET请求
  处理请求（req.params）
})

app.get('/skiers/:skierID', function (req, res) {
  // process the GET request
  ProcessRequest(req.params)
})

@RestController
公共类 SkierController {
     @GetMapping("/skiers/{skierID}",
                产生=“应用程序/json”）
    公共配置文件 GetSkierProfile(@PathVariable String skierID) {
        // 为了简洁省略了数据库查询方法
        返回 GetProfileFromDB(滑雪者ID);
    }
}

@RestController
public class SkierController {
     @GetMapping("/skiers/{skierID}", 
                produces = “application/json”)
    public Profile GetSkierProfile(@PathVariable String skierID) {
        // DB query method omitted for brevity
        return GetProfileFromDB(skierID);
    }
}

导入 javax.servlet.http.*;
@WebServlet(
    名称=“SkiersServlet”，
    urlPatterns =“/滑雪者”
）
公共类 SkierServlet 扩展了 HttpServlet (

protected void doGet(HttpServletRequest 请求,   
                     HttpServletResponse 响应）{  
  // 处理对 /skiers/{skierID} 的请求
  尝试 {
     // 从请求 URI 中提取 skierID（为简洁起见，未显示）
     String SkierID = getSkierIDFromRequest(request);    
     if(滑雪者ID == null) {  
        // 请求格式错误，返回错误代码
        response.setStatus(HttpServletResponse.SC_BAD_REQUEST); }
     别的 {      
        // 从数据库中读取滑雪者资料
        Profile profile = GetSkierProfile(skierID);
        // 将滑雪者个人资料作为 JSON 添加到 HTTP 响应并返回 200
        response.setContentType(“application/json”);
        response.getWriter().write(gson.toJson(Profile);
        response.setStatus(HttpServletResponse.SC_OK);
     } catch（异常前）{         
         响应.setStatus
           (HttpServletResponse.SC_INTERNAL_SERVER_ERROR); }
    
       }
} }

import javax.servlet.http.*;
@WebServlet(
    name = “SkiersServlet“,
    urlPatterns = “/skiers”
)
public class SkierServlet extends HttpServlet (

protected void doGet(HttpServletRequest request,   
                     HttpServletResponse response) {  
  // handles requests to /skiers/{skierID}
  try {
     // extract skierID from the request URI (not shown for brevity)
     String skierID  = getSkierIDFromRequest(request);    
     if(skierID == null) {  
        // request was poorly formatted, return error code
        response.setStatus(HttpServletResponse.SC_BAD_REQUEST);    }
     else {      
        // read the skier profile from the database 
        Profile profile = GetSkierProfile (skierID);
        // add skier profile as JSON to HTTP response and return 200
        response.setContentType("application/json");
        response.getWriter().write(gson.toJson(Profile);
        response.setStatus(HttpServletResponse.SC_OK); 
     } catch(Exception ex) {         
         response.setStatus
           (HttpServletResponse.SC_INTERNAL_SERVER_ERROR);    }
    
       }
} }

状态管理

State Management

状态管理是一个棘手的、微妙的话题。底端路线是需要扩展的服务实现应该避免存储会话状态。

State management is a tricky, nuanced topic. The bottom line is that service implementations that need to scale should avoid storing conversational state.

这到底是什么意思？让我们首先研究一下使用 HTTP 进行状态管理的主题。

What on earth does that mean? Let’s start by examining the topic of state management with HTTP.

HTTP 被称为无状态协议。这意味着每个请求都是独立执行的，不知道同一客户端之前执行的请求。无状态意味着每个请求都需要是独立的，客户端为 Web 服务器提供足够的信息来满足请求，无论该客户端之前的活动如何。

HTTP is known as stateless protocol. This means each request is executed independently, without any knowledge of the requests that were executed before it from the same client. Statelessness implies that every request needs to be self-contained, with sufficient information provided by the client for the web server to satisfy the request regardless of previous activity from that client.

然而，情况比这个简单的描述要复杂一些。例如：

The picture is a little more complicated that this simple description portrays, however. For example:

• 客户端和服务器之间的底层套接字连接保持打开状态，以便在客户端的多个请求之间分摊连接创建的开销。这是 HTTP/1 及更高版本的默认行为。

• The underlying socket connection between a client and server is kept open so that the overheads of connection creation are amortized across multiple requests from a client. This is the default behavior for versions HTTP/1 and above.

• HTTP 支持 cookies，称为HTTP 状态管理机制。光看名字就知道了，真的！

• HTTP supports cookies, which are known as the HTTP State Management Mechanism. The name gives it away, really!

• HTTP/2 支持流、压缩和加密，所有这些都需要状态管理。

• HTTP/2 supports streams, compression, and encryption, all of which require state management.

那么，最初 HTTP 是无状态的，但现在也许不再是无状态的了？带着这种困惑，我将继续讨论基于 HTTP 构建的应用程序服务 API 中的状态管理。

So, originally HTTP was stateless, but perhaps not anymore? Armed with this confusion, I’ll move on to state management in application services APIs that are built on top of HTTP.

当用户或应用程序连接到服务时，它通常会发送一系列请求来检索和更新信息。会话状态代表任何信息在请求之间保留，以便后续请求可以假设服务已保留有关先前交互的知识。我将通过一个简单的例子来探讨这意味着什么。

When a user or application connects to a service, it will typically send a series of requests to retrieve and update information. Conversational state represents any information that is retained between requests such that a subsequent request can assume the service has retained knowledge about the previous interactions. I’ll explore what this means in a simple example.

在滑雪者服务 API 中，用户可以通过GET向以下 URI 提交请求来请求其个人资料：

In the skier service API, a user may request their profile by submitting a GET request to the following URI:

获取/skico.com/skiers/768934

GET /skico.com/skiers/768934

然后，他们可以使用他们的应用程序修改其city属性并发送PUT更新资源的请求：

They may then use their app to modify their city attribute and send a PUT request to update the resource:

放置 /skico.com/skiers/
{
    “用户名”：“Ian123”，
    “电子邮件”：“i.gorton@somewhere.com”
    “城市”：“韦纳奇”
}

PUT /skico.com/skiers/
{
    "username": "Ian123",
    "email": "i.gorton@somewhere.com"
    "city": "Wenatchee"
}

由于此 URI 无法识别滑雪者，因此服务必须知道要更新的资源的唯一标识符，即 768934。因此，为了使此更新操作成功，服务必须保留先前请求的会话状态GET。

As this URI does not identify the skier, the service must know the unique identifier of the resource to update, namely 768934. Hence, for this update operation to succeed, the service must have retained conversational state from the previous GET request.

实施这种方法相对简单。当服务收到初始GET请求时，它会创建一个唯一标识客户端连接的会话状态对象。实际上，这通常在用户首次连接或登录服务时执行。然后，该服务可以从数据库中读取滑雪者资料，并利用会话状态对象来存储对话状态 - 在我们的示例中，这可能是skierID与滑雪者资料相关联的值。当后续PUT请求从客户端到达时，它使用会话状态对象来查找skierID与此会话关联的对象，并使用它来更新滑雪者的家乡城市。

Implementing this approach is relatively straightforward. When the service receives the initial GET request, it creates a session state object that uniquely identifies the client connection. In reality, this is often performed when a user first connects to or logs in to a service. The service can then read the skier profile from the database and utilize the session state object to store conversational state—in our example this would be skierID and likely values associated with the skier profile. When the subsequent PUT request arrives from the client, it uses the session state object to look up the skierID associated with this session and uses that to update the skier’s home city.

维护会话状态的服务称为有状态服务。从设计角度来看，有状态服务很有吸引力，因为它们可以最大限度地减少服务从数据库检索数据（状态）的次数，并减少客户端和服务之间传递的数据量。

Services that maintain conversational state are known as stateful services. Stateful services are attractive from a design perspective as they can minimize the number of times a service retrieves data (state) from the database and reduce the amount of data that is passed between clients and the services.

对于请求负载较轻的服务来说，它们具有显着的意义，并且被许多框架所推广，以使服务易于构建和部署。例如，JEE servlet 支持使用该HttpSession对象进行会话管理，并且SessionASP.NET 中的对象也提供了类似的功能。

For services with light request loads, they make eminent sense and are promoted by many frameworks to make services easy to build and deploy. For example, JEE servlets support session management using the HttpSession object, and similar capabilities are offered by the Session object in ASP.NET.

然而，当您扩展服务实现时，有状态方法就会出现问题。对于单个服务实例，您需要考虑两个问题：

As you scale the service implementations however, the stateful approach becomes problematic. For a single service instance, you have two problems to consider:

• 如果您有多个客户端会话都维护会话状态，这将利用可用的服务内存。使用的内存量将与服务维护状态的客户端数量成正比。如果请求突然激增，您如何确定我们不会耗尽可用内存并导致服务失败？

• If you have multiple client sessions all maintaining session state, this will utilize available service memory. The amount of memory utilized will be proportional to the number of clients the service is maintaining state for. If a sudden spike of requests arrives, how can you be certain we will not exhaust available memory and cause the service to fail?

• 您还必须注意保持会话状态可用的时间。客户端可能会停止发送请求，但不会完全关闭其连接以允许回收状态。所有会话管理方法都支持默认会话超时。如果将此时间间隔设置为较短的时间间隔，客户端可能会发现其状态意外消失。如果您将会话超时时间设置得太长，则可能会由于资源不足而降低服务性能。

• You also must be mindful about how long to keep session state available. A client may stop sending requests but not cleanly close their connection to allow the state to be reclaimed. All session management approaches support a default session timeout. If you set this to a short time interval, clients may see their state disappear unexpectedly. If you set the session timeout period to be too long, you may degrade service performance as it runs low on resources.

相反，无状态服务不假设先前调用的任何会话状态都已保留。该服务不应保留早期请求的任何知识，以便可以单独处理每个请求。这要求客户端提供服务处理请求并提供响应的所有必要信息。这实际上就是表 5-1中滑雪者 API 的指定方式，即：

In contrast, stateless services do not assume that any conversational state from previous calls has been preserved. The service should not maintain any knowledge from earlier requests, so that each request can be processed individually. This requires the client to provide all the necessary information for the service to process the request and provide a response. This is in fact how the skier API is specified in Table 5-1, namely:

放置 /skico.com/skiers/768934
{
    “用户名”：“Ian123”，
    “电子邮件”：“i.gorton@somewhere.com”
    “城市”：“韦纳奇”
}

PUT /skico.com/skiers/768934
{
    "username": "Ian123",
    "email": "i.gorton@somewhere.com"
    "city": "Wenatchee"
}

图 5-2显示了说明这种无状态设计的序列图。

A sequence diagram illustrating this stateless design is shown in Figure 5-2.

图 5-2。无状态 API 示例

任何可扩展的服务都将需要无状态 API。当我在本章后面解释水平缩放时，原因就会变得清楚。目前，最重要的设计含义是，对于需要保留与客户端会话相关的状态的服务（经典的购物车示例），该状态必须存储在服务外部。这无一例外指外部数据存储。

Any scalable service will need stateless APIs. The reason why will become clear when I explain horizontal scaling later in this chapter. For now, the most important design implication is that for a service that needs to retain state pertaining to client sessions—the classic shopping cart example—this state must be stored externally to the service. This invariably means an external data store.

应用服务器

Applications Servers

应用程序服务器是可扩展应用程序的核心，托管组成应用程序的业务服务应用。它们的基本作用是接受客户端的请求，将应用程序逻辑应用于请求，并将请求结果回复给客户端。客户端可以是外部的或内部的，就像应用程序中需要使用特定服务的功能的其他服务一样。

Application servers are the heart of a scalable application, hosting the business services that compose an application. Their basic role is to accept requests from clients, apply application logic to the requests, and reply to the client with the request results. Clients may be external or internal, as in other services in the application that require to use the functionality of a specific service.

应用服务器的技术前景广泛而复杂，具体取决于您要使用的语言以及每种语言提供的特定功能。在 Java 中，Java Enterprise Edition (JEE)为应用程序服务器定义了一个全面的、功能丰富的、基于标准的平台，具有多个不同的供应商和开源实现。

The technological landscape of application servers is broad and complex, depending on the language you want to use and the specific capabilities that each offers. In Java, the Java Enterprise Edition (JEE) defines a comprehensive, feature rich, standards-based platform for application servers, with multiple different vendor and open source implementations.

在其他语言中，Express.js服务器支持 Node，Flask支持 Python，Go 中支持服务可以通过合并包来创建net/http。这些实现比 JEE 更​​加简约和轻量，通常被归类为 Web 应用程序框架。在 Java 中，Apache Tomcat 服务器是一个某种程度相当的技术。Tomcat 是一个开源实现JEE 平台的子集，即 Java servlet、JavaServer Pages (JSP)、Java 表达式语言 (EL) 和 Java WebSocket 技术。

In other languages, the Express.js server supports Node, Flask supports Python, and in Go a service can be created by incorporating the net/http package. These implementations are much more minimal and lightweight than JEE and are typically classified as web application frameworks. In Java, the Apache Tomcat server is a somewhat equivalent technology. Tomcat is an open source implementation of a subset of the JEE platform, namely the Java servlet, JavaServer Pages (JSP), Java Expression Language (EL), and Java WebSocket technologies.

图 5-3描绘了 Tomcat 的简化剖视图。Tomcat 实现了servlet 容器，它是应用程序定义的 servlet 的执行环境。Servlet 动态加载到该容器中，该容器提供生命周期管理和多线程运行时环境。

Figure 5-3 depicts a simplified view of the anatomy of Tomcat. Tomcat implements a servlet container, which is an execution environment for application-defined servlets. Servlets are dynamically loaded into this container, which provides life cycle management and a multithreaded runtime environment.

图 5-3。Web 应用程序服务器剖析

请求到达服务器的 IP 地址，该服务器正在侦听特定端口上的流量。例如，默认情况下，Tomcat 在端口 8080 上侦听 HTTP 请求，在端口 8443 上侦听 HTTPS 请求。传入请求由一个或多个侦听器线程处理。它们在客户端和服务器之间创建 TCP/IP 套接字连接。如果网络请求到达的频率无法被 TCP 侦听器处理，则挂起的请求将在Sockets Backlog中排队。积压的大小取决于操作系统。在大多数 Linux 版本中，默认值为 100。

Requests arrive at the IP address of the server, which is listening for traffic on specific ports. For example, by default Tomcat listens on port 8080 for HTTP requests and 8443 for HTTPS requests. Incoming requests are processed by one or more listener threads. These create a TCP/IP socket connection between the client and server. If network requests arrive at a frequency that cannot be processed by the TCP listener, pending requests are queued up in the Sockets Backlog. The size of the backlog is operating system dependent. In most Linux versions the default is 100.

建立连接后，TCP 请求将由HTTP 连接器（在本示例中）进行编组，该连接器生成Servlet 可以处理的HTTP 请求（如图5-2HttpServletRequest中的对象）。然后，HTTP 请求被分派到应用程序容器线程进行处理。

Once a connection is established, the TCP requests are marshalled by, in this example, an HTTP Connector which generates the HTTP request (HttpServletRequest object as in Figure 5-2) that the servlet can process. The HTTP request is then dispatched to an application container thread to process.

应用程序容器线程是在线程池中进行管理，本质上是一个 Java Executor，默认情况下，Tomcat 中的最小大小为 25 个线程，最大为 200 个。如果没有可用线程来处理请求，容器会将它们维护在可运行任务的队列中，一旦线程可用，就会调度它们。默认情况下，该队列的大小是Integer.MAX_VALUE无限的。³默认情况下，如果线程空闲 60 秒，则会将其杀死以释放 Java 虚拟机中的资源。

Application container threads are managed in a thread pool, essentially a Java Executor, which by default in Tomcat is a minimum size of 25 threads and a maximum of 200. If there are no available threads to handle a request, the container maintains them in a queue of runnable tasks and dispatches these as soon as a thread becomes available. This queue by default is size Integer.MAX_VALUE—that is, essentially unbounded.³ By default, if a thread remains idle for 60 seconds, it is killed to free up resources in the Java virtual machine.

对于每个请求，都会在线程中调用与 HTTP 请求对应的方法。Servlet 方法处理 HTTP 请求标头、执行业务逻辑并构造响应，该响应由容器编组回 TCP/IP 数据包并通过网络发送到客户端。

For each request, the method that corresponds with the HTTP request is invoked in a thread. The servlet method processes the HTTP request headers, executes the business logic, and constructs a response that is marshalled by the container back into a TCP/IP packet and sent over the network to the client.

在处理业务逻辑时，Servlet 经常需要查询外部数据库。这需要每个执行 servlet 方法的线程获取数据库连接并执行数据库查询。在许多数据库中，尤其是关系数据库中，连接是有限的资源，因为它们消耗客户端和数据库服务器中的内存和系统资源。因此，通常使用固定大小的数据库连接池。该池按需向请求线程分发打开的连接。

In processing the business logic, servlets often need to query an external database. This requires each thread executing the servlet methods to obtain a database connection and execute database queries. In many databases, especially relational ones, connections are limited resources as they consume memory and system resources in both the client and database server. For this reason, a fixed-size database connection pool is typically utilized. The pool hands out open connections to requesting threads on demand.

当 servlet 希望向数据库提交查询时，它会从池中请求连接。如果可用，则向 servlet 授予对连接的访问​​权限，直到它指示它已完成其工作。在此阶段，连接将返回到池中并可供另一个 servlet 使用。由于容器线程池通常大于数据库连接池，因此 servlet 可能会在没有可用连接时请求连接。为了处理这个问题，连接池维护一个请求队列，并以 FIFO 为基础分发打开的连接，队列中的线程将被阻塞，直到可用或发生超时。

When a servlet wishes to submit a query to the database, it requests a connection from the pool. If one is available, access to the connection is granted to the servlet until it indicates it has completed its work. At that stage the connection is returned to the pool and made available for another servlet to utilize. As the container thread pool is typically larger than the database connection pool, a servlet may request a connection when none are available. To handle this, the connection pool maintains a request queue and hands out open connections on a FIFO basis, and threads in the queue are blocked until there is availability or a timeout occurs.

因此，诸如 Tomcat 之类的应用程序服务器框架对于不同的工作负载具有高度可配置性。例如，可以在启动时读取的配置文件中指定线程和数据库连接池的大小。

An application server framework such as Tomcat is hence highly configurable for different workloads. For example, the size of the thread and database connection pools can be specified in configuration files that are read at startup.

完整的 Tomcat 容器环境在单个 JVM 中运行，因此处理能力受到可用 vCPU 数量和分配为堆大小的内存量的限制。每个分配的线程都会消耗内存，并且请求处理管道中的各个队列会在请求等待时消耗资源。这意味着请求响应时间将由 servlet 业务逻辑中的请求处理时间以及在队列中等待线程和连接可用的时间决定。

The complete Tomcat container environment runs within a single JVM, and hence processing capacity is limited by the number of vCPUs available and the amount of memory allocated as heap size. Each allocated thread consumes memory, and the various queues in the request-processing pipeline consume resources while requests are waiting. This means that request response time will be governed by both the request-processing time in the servlet business logic as well as the time spent waiting in queues for threads and connections to become available.

在分配了许多线程的负载较重的服务器中，上下文切换可能会开始降低性能，并且可用内存可能会变得有限。如果性能下降，队列会随着请求等待资源而增长。这会消耗更多内存。如果收到的请求多于服务器可以排队和处理的请求，则新的 TCP/IP 连接将被拒绝，并且客户端将看到错误。最终，过载的服务器可能会耗尽资源并开始抛出异常并崩溃。

In a heavily loaded server with many threads allocated, context switching may start to degrade performance, and available memory may become limited. If performance degrades, queues grow as requests wait for resources. This consumes more memory. If more requests are received than can be queued up and processed by the server, then new TCP/IP connections will be refused, and clients will see errors. Eventually, an overloaded server may run out of resources and start throwing exceptions and crash.

因此，花在调整配置参数以有效处理预期负载上的时间很少被浪费。系统在达到 100% 利用率之前，性能往往会下降。一旦任何资源（CPU 利用率、内存使用率、网络、磁盘访问等）接近完全利用率，系统就会表现出难以预测的性能。这是因为更多的时间花费在线程上下文切换和内存垃圾收集等浪费时间的任务上。这不可避免地会影响延迟和吞吐量。因此，有一个利用目标是至关重要的。这些阈值到底应该是多少，很大程度上取决于应用程序。

Consequently, time spent tuning configuration parameters to efficiently handle anticipated loads is rarely wasted. Systems tend to degrade in performance well before they reach 100% utilization. Once any resource—CPU utilization, memory usage, network, disk accesses, etc.—gets close to full utilization, systems exhibit less predictable performance. This is because more time is spent on time-wasting tasks such as thread context switching and memory garbage collecting. This inevitably affects latencies and throughput. Thus, having a utilization target is essential. Exactly what these thresholds should be is extremely application dependent.

Web 应用程序框架提供的监控工具使工程师能够收集一系列重要指标，包括延迟、活动请求、队列大小等。这些对于进行可实现性能优化的数据驱动实验非常宝贵。

Monitoring tools available with web application frameworks enable engineers to gather a range of important metrics, including latencies, active requests, queue sizes, and so on. These are invaluable for carrying out data-driven experiments that lead to performance optimization.

基于 Java 的应用程序框架（例如 Tomcat）支持Java 管理扩展 (JMX) 框架，该框架是 Java 标准版平台的标准部分。JMX 使框架能够基于 MBean 的功能公开监控信息，这些信息代表感兴趣的资源（例如，线程池、数据库连接使用情况）。这使得工具生态系统能够提供监视 JMX 支持的平台的功能。这些范围从默认情况下在 Java 开发工具包 (JDK) 中提供的JConsole到强大的开源技术（例如JavaMelody）和许多昂贵的商业产品。

Java-based application frameworks such as Tomcat support the Java Management Extensions (JMX) framework, which is a standard part of the Java Standard Edition platform. JMX enables frameworks to expose monitoring information based on the capabilities of MBeans, which represent a resource of interest (e.g., thread pool, database connections usage). This enables an ecosystem of tools to offer capabilities for monitoring JMX-supported platforms. These range from JConsole, which is available in the Java Development Kit (JDK) by default, to powerful open source technologies such as JavaMelody and many expensive commercial offerings.

水平缩放

Horizontal Scaling

扩展系统的核心原则是能够轻松添加新的处理能力来处理增加的加载。对于大多数系统，一种简单有效的方法是部署无状态服务器资源的多个实例，并使用负载均衡器在这些实例之间分配请求。这称为水平缩放，如图5-4所示。无状态服务副本和负载均衡器都是水平扩展所必需的。

A core principle of scaling a system is being able to easily add new processing capacity to handle increased load. For most systems, a simple and effective approach is deploying multiple instances of stateless server resources and using a load balancer to distribute the requests across these instances. This is known as horizontal scaling and is illustrated in Figure 5-4. Stateless service replicas and a load balancer are both necessary for horizontal scaling.

图 5-4。简单的负载均衡示例

服务副本已部署在他们自己的（虚拟）硬件上。如果我们有两个副本，我们的处理能力就会加倍。如果我们有 10 个副本，我们就有可能拥有 10 倍的容量。这使我们的系统能够处理增加的负载。水平扩展的目的是创建系统处理能力，即可用总资源的总和。

Service replicas are deployed on their own (virtual) hardware. If we have two replicas, we double our processing capacity. If we have ten replicas, we have potentially 10x capacity. This enables our system to handle increased loads. The aim of horizontal scaling is to create a system-processing capacity that is the sum of the total resources available.

服务器需要是无状态的，以便任何请求都可以发送到任何服务副本来处理。该决定由负载均衡器做出，负载均衡器可以使用各种策略来分发请求。如果负载均衡器可以使每个服务副本保持同等繁忙，那么我们就可以有效地利用服务副本提供的处理能力。

The servers need to be stateless, so that any request can be sent to any service replica to handle. This decision is made by the load balancer, which can use various policies to distribute requests. If the load balancer can keep each service replica equally busy, then we are effectively using the processing capacity provided by the service replicas.

如果我们的服务是有状态的，负载均衡器需要始终将来自同一服务器的请求路由到相同的服务副本。由于客户端会话的持续时间不确定，这可能会导致某些副本的负载比其他副本重得多。这会造成不平衡，并且无法有效地跨副本均匀地使用可用容量。我将在下一节有关负载平衡的内容中更详细地讨论这个问题。

If our services are stateful, the load balancer needs to always route requests from the same server to the same service replica. As client sessions have indeterminate durations, this can lead to some replicas being much more heavily loaded than others. This creates an imbalance and is not effective in using the available capacity evenly across replicas. I’ll return to this issue in more detail in the next section on load balancing.

水平扩展还提高了可用性。对于一个服务实例，如果失败，则该服务不可用。这被称为单点故障（SPoF） ——这是一件坏事，在任何可扩展的分布式系统中都应该避免。多个副本可提高可用性。如果一个副本发生故障，请求可以定向到任何副本 - 请记住，它们是无状态的。在更换故障服务器之前，系统的容量将减少，但仍然可用。可扩展的能力至关重要，但如果系统不可用，那么有史以来最具可扩展性的系统建起来还是有点效果！

Horizontal scaling also increases availability. With one service instance, if it fails, the service is unavailable. This is known as a single point of failure (SPoF)—a bad thing, and something to avoid in any scalable distributed system. Multiple replicas increase availability. If one replica fails, requests can be directed to any replica—remember, they are stateless. The system will have reduced capacity until the failed server is replaced, but it will still be available. The ability to scale is crucial, but if a system is unavailable, then the most scalable system ever built is still somewhat ineffective!

负载均衡

Load Balancing

负载均衡旨在有效利用服务集合优化每个请求的响应时间的能力。这是通过在可用服务之间分配请求以理想地利用集体服务能力来实现的。目的是避免某些服务过载而其他服务利用率不足。

Load balancing aims to effectively utilize the capacity of a collection of services to optimize the response time for each request. This is achieved by distributing requests across the available services to ideally utilize the collective service capacity. The objective is to avoid overloading some services while underutilizing others.

客户端将请求发送到负载均衡器的 IP 地址，负载均衡器将请求重定向到目标服务，并将结果转发回客户端。这意味着客户端永远不会直接联系目标服务，这也有利于安全，因为服务可以位于安全边界后面并且不会暴露在互联网上。

Clients send requests to the IP address of the load balancer, which redirects requests to target services, and relays the results back to the client. This means clients never contact the target services directly, which is also beneficial for security as the services can live behind a security perimeter and not be exposed to the internet.

负载均衡器可能会起作用在网络级别或应用程序级别。它们通常分别称为第 4 层和第 7 层负载均衡器。这些名称指的是开放系统互连 (OSI) 参考模型中第 4 层的网络传输层和第 7 层的应用层。OSI 模型在七个抽象层中定义了网络功能。每层都定义了数据打包和传输方式的标准。

Load balancers may act at the network level or the application level. These are often called layer 4 and layer 7 load balancers, respectively. The names refer to network transport layer at layer 4 in the Open Systems Interconnection (OSI) reference model, and the application layer at layer 7. The OSI model defines network functions in seven abstract layers. Each layer defines standards for how data is packaged and transported.

网络级负载均衡器在网络连接级别分发请求，对单个 TCP 或 UDP 数据包进行操作。路由决策是根据客户端 IP 地址做出的。一旦选择了目标服务，负载均衡器就会使用一种技术称为网络地址转换（NAT）。这会将客户端请求数据包中的目标 IP 地址从负载均衡器的地址更改为所选目标的地址。当从目标接收到响应时，负载均衡器将数据包标头中记录的源地址从目标的 IP 地址更改为自己的 IP 地址。网络负载均衡器相对简单，因为它们在单个数据包级别上运行。这意味着它们的速度非常快，因为除了选择目标服务和执行 NAT 功能之外，它们提供的功能很少。

Network-level load balancers distribute requests at the network connection level, operating on individual TCP or UDP packets. Routing decisions are made on the basis of client IP addresses. Once a target service is chosen, the load balancer uses a technique called network address translation (NAT). This changes the destination IP address in the client request packet from that of the load balancer to that of the chosen target. When a response is received from the target, the load balancer changes the source address recorded in the packet header from the target’s IP address to its own. Network load balancers are relatively simple as they operate on the individual packet level. This means they are extremely fast, as they provide few features beyond choosing a target service and performing NAT functionality.

相比之下，应用程序级负载均衡器重新组装完整的 HTTP 请求，并根据 HTTP 标头的值和消息的实际内容做出路由决策。例如，负载均衡器可以配置为将所有POST请求发送到可用服务的子集，或根据 URI 中的查询字符串分发请求。应用程序负载均衡器是复杂的反向代理。它们提供的更丰富的功能意味着它们比网络负载均衡器稍慢，但它们提供的强大功能可以用来弥补所产生的开销。

In contrast, application-level load balancers reassemble the complete HTTP request and base their routing decisions on the values of the HTTP headers and on the actual contents of the message. For example, a load balancer can be configured to send all POST requests to a subset of available services, or distribute requests based on a query string in the URI. Application load balancers are sophisticated reverse proxies. The richer capabilities they offer means they are slightly slower than network load balancers, but the powerful features they offer can be utilized to more than make up for the overheads incurred.

为了让您了解网络层和应用程序层负载均衡器之间的原始性能差异，图 5-7在一个简单的应用场景中对两者进行了比较。正在测试的负载均衡技术是AWS应用程序和网络弹性负载均衡器（ELB）。每个负载均衡器路由请求到 4 个副本之一。它们执行业务逻辑并通过负载均衡器将结果返回给客户端。客户端负载从轻负载的 32 个并发客户端到中等负载的 256 个并发客户端不等。每个客户端发送一系列请求，在接收一个请求的结果和向服务器发送下一个请求之间没有延迟。

To give you some idea of the raw performance difference between network- and application-layer load balancers, Figure 5-7 compares the two in a simple application scenario. The load balancing technology under test is the AWS Application and Network Elastic Load Balancers (ELBs). Each load balancer routes requests to one of 4 replicas. These execute the business logic and return results to the clients via the load balancer. Client load varies from a lightly loaded 32 concurrent clients to a moderate 256 concurrent clients. Each client sends a sequence of requests with no delay between receiving the results from one request and sending the next request to the server.

从图 5-5中可以看出，对于 32、64 和 128 个客户端测试，网络负载均衡器的性能平均提高了约 20%。这验证了不太复杂的网络负载均衡器预期的更高性能。对于 256 个客户端，两个负载均衡器的性能基本相同。这是因为超过了4个副本的容量，系统出现瓶颈。在此阶段，负载均衡器对系统性能没有影响。您需要向负载均衡组添加更多副本以增加系统容量，从而提高吞吐量。

You can see from Figure 5-5 that the network load balancer delivers on average around 20% higher performance for the 32, 64, and 128 client tests. This validates the expected higher performance from the less sophisticated network load balancer. For 256 clients, the performance of the two load balancers is essentially the same. This is because the capacity of the 4 replicas is exceeded and the system has a bottleneck. At this stage the load balancers make no difference to the system performance. You need to add more replicas to the load balancing group to increase system capacity, and hence throughput.

图 5-5。比较负载均衡器性能⁴

一般来说，负载均衡器具有以下功能，将在以下部分中进行解释：

In general, a load balancer has the following features that will be explained in the following sections:

• 负载分配策略

• Load distribution policies

• 健康监测

• Health monitoring

• 弹性

• Elasticity

• 会话关联性

• Session affinity

弹性

Elasticity

请求负载的峰值可能会导致负载均衡器可用的服务容量变得饱和，导致响应时间更长，最终导致请求和连接失败。弹性是应用程序动态提供新服务容量以处理请求增加的能力。随着负载的增加，新的副本将启动，负载均衡器会将请求定向到这些副本。随着负载减少，负载均衡器会停止不再需要的服务。

Spikes in request loads can cause the service capacity available to a load balancer to become saturated, leading to longer response times and eventually request and connection failures. Elasticity is the capability of an application to dynamically provision new service capacity to handle an increase in requests. As load increases, new replicas are started and the load balancer directs requests to these. As load decreases, the load balancer stops services that are no longer needed.

弹性要求负载均衡器与应用程序监控紧密集成，以便可以定义扩展策略来确定何时扩展和缩减。例如，策略可以指定，当所有实例的平均服务 CPU 利用率超过 70% 时，应增加服务的容量，而当平均 CPU 利用率低于 40% 时，应减少服务的容量。通常可以使用通过监控系统可用的任何指标来定义扩展策略。

Elasticity requires a load balancer to be tightly integrated with application monitoring, so that scaling policies can be defined to determine when to scale up and down. Policies may specify, for example, that capacity for a service should be increased when the average service CPU utilization across all instances is over 70%, and decreased when average CPU utilization is below 40%. Scaling policies can typically be defined using any metrics that are available through the monitoring system.

弹性负载平衡的一个示例是 AWS Auto Scaling 组。Auto Scaling 组是可用于定义了最小和最大大小的负载均衡器的服务实例的集合。负载均衡器将确保该组始终具有最少数量的可用服务，并且该组永远不会超过最大数量。该方案如图5-6所示。

An example of elastic load balancing is the AWS Auto Scaling groups. An Auto Scaling group is a collection of service instances available to a load balancer that is defined with a minimum and maximum size. The load balancer will ensure the group always has the minimum numbers of services available, and the group will never exceed the maximum number. This scheme is illustrated in Figure 5-6.

图 5-6。弹性负载均衡

通常，有两种方法可以控制组中的副本数量。第一个基于时间表，请求负载何时增加和减少是可预测的。例如，您可能有一个在线娱乐指南，并在周四下午 6 点发布一组主要城市的周末活动。这会在周日中午之前产生更高的负载。Auto Scaling 组可以轻松配置为在周四下午 6 点提供新服务，并在周日中午将组大小减少到最小。

Typically, there are two ways to control the number of replicas in a group. The first is based on a schedule, when the request load increases and decreases are predictable. For example, you may have an online entertainment guide and publish the weekend events for a set of major cities at 6 p.m. on Thursday. This generates a higher load until Sunday at noon. An Auto Scaling group could easily be configured to provision new services at 6 p.m. Thursday and reduce the group size to the minimum at noon Sunday.

如果增加的负载峰值不可预测，则可以通过基于应用程序指标（例如平均 CPU 和内存使用情况以及队列中的消息数量）定义的扩展策略来动态控制弹性。如果超过策略的上限阈值，负载均衡器将启动一个或多个新的服务实例，直到性能低于指标阈值。实例需要时间来启动（通常是一分钟或更长时间），因此可以定义一个预热期，直到新实例被认为对组的容量做出了贡献。当观察到的指标值低于扩展策略中定义的下限阈值时，就会开始缩小或缩小，实例将自动停止并从池中删除。

If increased load spikes are not predictable, elasticity can be controlled dynamically by defined scaling policies based on application metrics such as average CPU and memory usage and number of messages in a queue. If the upper threshold of the policy is exceeded, the load balancer will start one or more new service instances until performance drops below the metric threshold. Instances need time to start—often a minute or more—and hence a warm-up period can be defined until the new instance is considered to be contributing to the group’s capacity. When the observed metric value drops below the lower threshold defined in the scaling policy, scale in or scale down commences and instances will be automatically stopped and removed from the pool.

弹性是一项关键功能，它允许服务随着需求的增长而动态扩展。对于工作负载波动的高度可扩展的系统，它几乎是提供以最低成本提供必要的能力。

Elasticity is a key feature that allows services to scale dynamically as demand grows. For highly scalable systems with fluctuating workloads, it is pretty much a mandatory capability for providing the necessary capacity at minimum costs.

会话亲和性

Session Affinity

会话亲和性或粘性会话是有状态服务的负载平衡器功能。通过粘性会话，负载均衡器将来自同一客户端的所有请求发送到同一服务实例。这使得服务能够维护每个特定客户端会话的内存状态。

Session affinity, or sticky sessions, are a load balancer feature for stateful services. With sticky sessions, the load balancer sends all requests from the same client to the same service instance. This enables the service to maintain in-memory state about each specific client session.

有多种方法可以实现粘性会话。例如，HAProxy 提供了一套全面的功能，可以在服务添加、删除和故障时维护对同一服务的客户端请求。AWS Elastic Load Balancing (ELB) 生成一个 HTTP cookie，用于标识客户端会话的服务副本与....关联。该 cookie 返回给客户端，客户端必须在后续请求中发送该 cookie，以确保维持会话关联性。

There are various ways to implement sticky sessions. For example, HAProxy provides a comprehensive set of capabilities to maintain client requests on the same service in the face of service additions, removals, and failures. AWS Elastic Load Balancing (ELB) generates an HTTP cookie that identifies the service replica a client’s session is associated with. This cookie is returned to the client, which must send it in subsequent requests to ensure session affinity is maintained.

对于高度可扩展的系统来说，粘性会话可能会出现问题。它们会导致负载不平衡问题，随着时间的推移，客户端在服务之间的分布不均匀。如图 5-7所示，其中两个客户端连接到一个服务，而另一个服务保持空闲。

Sticky sessions can be problematic for highly scalable systems. They lead to a load imbalance problem, in which, over time, clients are not evenly distributed across services. This is illustrated in Figure 5-7, where two clients are connected to one service while another service remains idle.

图 5-7。会话粘性导致负载不平衡

由于客户端会话持续的时间不同，因此会出现负载不平衡。即使会话最初均匀分布，有些会话也会很快终止，而另一些会话则会持续存在。在轻负载系统中，这往往不是问题。然而，在一个不断创建和销毁数百万个会话的系统中，负载不平衡是不可避免的。这将导致一些服务副本未得到充分利用，而另一些服务副本则不堪重负，并可能因资源耗尽而失败。为了帮助缓解负载不平衡，负载均衡器通常提供诸如将新会话发送到连接数最少或响应时间最快的实例等策略。这些有助于引导新会话远离负载过重的服务。

Load imbalance occurs because client sessions last for varying amounts of time. Even if sessions are evenly distributed initially, some will terminate quickly while others will persist. In a lightly loaded system, this tends to not be an issue. However, in a system with millions of sessions being created and destroyed constantly, load imbalance is inevitable. This will lead to some service replicas being underutilized, while others are overwhelmed and may potentially fail due to resource exhaustion. To help alleviate load imbalance, load balancers usually provide policies such as sending new sessions to instances with the least connections or fastest response times. These help direct new sessions away from heavily loaded services.

有状态服务还有其他缺点。当服务不可避免地失败时，连接到该服务器的客户端如何恢复正在管理的状态？如果服务实例由于高负载而变慢，客户端如何响应？一般来说，有状态服务器会产生一些问题，在大型系统中这些问题可能难以设计和管理。

Stateful services have other downsides. When a service inevitably fails, how do the clients connected to that server recover the state that was being managed? If a service instance becomes slow due to high load, how do clients respond? In general, stateful servers create problems that in large scale systems can be difficult to design around and manage.

无状态服务没有这些缺点。如果失败，客户端会收到异常并重试，并将其请求路由到另一个实时服务副本。如果服务由于短暂的网络中断而速度缓慢，负载均衡器会将其从服务组中删除，直到它通过健康检查或失败。所有应用程序状态要么外部化，要么由客户端在每个请求中提供，因此负载均衡器可以轻松处理服务故障。

Stateless services have none of these downsides. If one fails, clients get an exception and retry, with their request routed to another live service replica. If a service is slow due to a transient network outage, the load balancer takes it out of the service group until it passes health checks or fails. All application state is either externalized or provided by the client in each request, so service failures can be handled easily by the load balancer.

无状态服务增强可扩展性，简化故障场景，减轻服务管理的负担。对于可扩展应用程序来说，这些优点远远超过缺点，因此它们被大多数主要的大型互联网站点（例如 Netflix）所采用。

Stateless services enhance scalability, simplify failure scenarios, and ease the burden of service management. For scalable applications, these advantages far outweigh the disadvantages, and hence their adoption in most major, large-scale internet sites such as Netflix.

最后，请记住，通过负载平衡扩展一组服务可能会淹没负载平衡服务所依赖的下游服务或数据库。就像高速公路一样，如果高速公路的终点是单车道的一组红绿灯，那么在 50 英里的时间内增加 8 条车道只会造成更大的交通混乱另一方面。我确信我们都去过那里。我将在第 9 章中解决这些问题。

Finally, bear in mind that scaling one collection of services through load balancing may well overwhelm downstream services or databases that the load balanced services depend on. Just like with highways, adding eight traffic lanes for 50 miles will just cause bigger traffic chaos if the highway ends at a set of traffic lights with a one-lane road on the other side. We’ve all been there, I’m sure. I’ll address these issues in Chapter 9.

总结和延伸阅读

Summary and Further Reading

服务是可扩展软件系统的核心。他们将合同定义为 API，向客户指定他们的功能。服务在托管服务代码并将传入 API 请求路由到适当的处理逻辑的应用程序服务器容器环境中执行。应用程序服务器高度依赖于编程语言，但通常提供多线程编程模型，允许服务同时处理许多请求。如果容器线程池中的线程全部被利用，应用服务器会将请求排队，直到有线程可用。

Services are the heart of a scalable software system. They define the contract as an API that specifies their capabilities to clients. Services execute in an application server container environment that hosts the service code and routes incoming API requests to the appropriate processing logic. Application servers are highly programming language dependent, but in general provide a multithreaded programming model that allows services to process many requests simultaneously. If the threads in the container thread pool are all utilized, the application server queues up requests until a thread becomes available.

随着服务上请求负载的增长，我们可以使用负载均衡器水平扩展它，以在多个实例之间分配请求。该架构还提供高可用性，因为多服务配置意味着应用程序可以容忍单个实例的故障。服务实例由负载均衡器作为池进行管理，负载均衡器利用负载分配策略为每个请求选择目标服务副本。无状态服务允许负载均衡器简单地向响应目标重新发送请求，从而轻松扩展并简化故障场景。尽管大多数负载均衡器将使用称为粘性会话的功能来支持有状态服务，但有状态服务使负载均衡和处理故障更加复杂。因此，不建议将它们用于高度可扩展的服务。

As request loads grow on a service, we can scale it out horizontally using a load balancer to distribute requests across multiple instances. This architecture also provides high availability as the multiple-service configuration means the application can tolerate failures of individual instances. The service instances are managed as a pool by the load balancer, which utilizes a load distribution policy to choose a target service replica for each request. Stateless services scale easily and simplify failure scenarios by allowing the load balancer to simply resend requests to responsive targets. Although most load balancers will support stateful services using a feature called sticky sessions, stateful services make load balancing and handling failures more complex. Hence, they are not recommended for highly scalable services.

Java Enterprise Edition (JEE) 是一种成熟且广泛部署的服务器端技术。它具有广泛的抽象，可用于构建丰富而强大的服务。Oracle教程是了解该平台的绝佳起点。

The Java Enterprise Edition (JEE) is an established and widely deployed server-side technology. It has a wide range of abstractions for building rich and powerful services. The Oracle tutorial is an excellent starting point for appreciating this platform.

有关负载均衡器的许多知识和信息都隐藏在技术供应商提供的文档中。您选择负载均衡器，然后深入阅读手册。Tony Bourke 所著的《Server Load Balancing》 （O'Reilly，2001 年）是一个很好的资源，可以让您对负载平衡的整个领域有一个出色、广泛的了解。

Much of the knowledge and information about load balancers is buried in the documentation provided by the technology suppliers. You choose your load balancer and then dive into the manuals. For an excellent, broad perspective on the complete field of load balancing, Server Load Balancing by Tony Bourke (O’Reilly, 2001) is a good resource.

应用程序缓存

Application Caching

应用程序缓存旨在通过将查询和计算的结果存储在内存中来提高请求响应能力，以便后续请求可以为它们提供服务。例如，考虑一个在线报纸网站，读者可以在其中发表评论。文章一旦发布，就很少更改（如果有的话）。新评论往往会在文章发表后很快发布，但随着文章时间的推移，频率会迅速下降。因此，一篇文章可以在第一次访问时缓存，并由所有后续请求重用，直到该文章被更新、发布新评论或没有人想再阅读它。

Application caching is designed to improve request responsiveness by storing the results of queries and computations in memory so they can be subsequently served by later requests. For example, think of an online newspaper site where readers can leave comments. Once posted, articles change infrequently, if ever. New comments tend to get posted soon after an article is published, but the frequency drops quickly with the age of the article. Hence an article can be cached on first access and reused by all subsequent requests until the article is updated, new comments are posted, or no one wants to read it anymore.

一般来说，缓存可以减轻数据库的大量读取流量，因为许多查询可以直接从缓存中提供服务。它还降低了构建成本高昂的对象的计算成本，例如那些需要跨越多个不同数据库的查询的对象。最终效果是减少我们服务和数据库的计算负载，并为更多请求创造空间或容量。

In general, caching relieves databases of heavy read traffic, as many queries can be served directly from the cache. It also reduces computation costs for objects that are expensive to construct, for example, those needing queries that span several different databases. The net effect is to reduce the computational load on our services and databases and create headroom, or capacity for more requests.

缓存需要额外的资源，因此需要额外的成本来存储缓存的结果。然而，与升级数据库和服务节点以应对更高的请求负载相比，精心设计的缓存方案成本较低。作为缓存价值的体现，Twitter 大约 3% 的基础设施专门用于应用程序级缓存。在 Twitter 规模上，运行数百个集群，这相当于大量的基础设施！

Caching requires additional resources, and hence cost, to store cached results. However, well-designed caching schemes are low cost compared to upgrading database and service nodes to cope with higher request loads. As an indication of the value of caches, approximately 3% of infrastructure at Twitter is dedicated to application-level caches. At Twitter scale, operating hundreds of clusters, that is a lot of infrastructure!

应用程序级缓存漏洞专用的分布式缓存引擎。该领域的两种主要技术是memcached和Redis。两者本质上都是分布式内存哈希表，专为表示数据库查询或下游服务 API 调用结果的任意数据（字符串、对象）而设计。缓存的常见用例是存储用户会话数据、动态网页和数据库查询结果。缓存对于应用程序服务来说就像一个单一的存储，并且使用对象键上的哈希函数将对象分配到各个缓存服务器。

Application-level caching exploits dedicated distributed cache engines. The two predominant technologies in this area are memcached and Redis. Both are essentially distributed in-memory hash tables designed for arbitrary data (strings, objects) representing the results of database queries or downstream service API calls. Common use cases for caches are storing user session data, dynamic web pages, and results of database queries. The cache appears to application services as a single store, and objects are allocated to individual cache servers using a hash function on the object key.

基本方案如图6-1所示。该服务首先检查缓存以查看其所需的数据是否可用。如果是，则返回缓存内容作为结果——这称为缓存命中。如果数据不在缓存中（缓存未命中），服务将从数据库检索请求的数据并将查询结果写入缓存，以便后续客户端请求可用，而无需查询数据库。

The basic scheme is shown in Figure 6-1. The service first checks the cache to see if the data it requires is available. If so, it returns the cached contents as the results—this is known as a cache hit. If the data is not in the cache—a cache miss—the service retrieves the requested data from the database and writes the query results to the cache so it is available for subsequent client requests without querying the database.

图 6-1。应用级缓存

例如，在繁忙的冬季度假胜地，滑雪者和单板滑雪者可以使用他们的移动应用程序来估计整个度假胜地的电梯等待时间。这使他们能够计划并避开拥挤的区域，因为在这些区域，他们必须等待 15 分钟或更长时间才能乘坐电梯！

For example, at a busy winter resort, skiers and snowboarders can use their mobile app to get an estimate of the lift wait times across the resort. This enables them to plan and avoid congested areas where they will have to wait to ride a lift for 15 minutes or more!

每次滑雪者装载缆车时，都会向该公司的服务发送一条消息，该服务收集有关滑雪者交通模式的数据。利用这些数据，系统可以根据乘坐缆车的滑雪者数量和到达率来估计缆车等待时间。这是一项昂贵的计算，在繁忙时间可能需要一秒钟或更长时间，因为它需要汇总可能数以万计的电梯乘坐记录并执行等待时间计算。因此，一旦计算出结果，它们就被认为在五分钟内有效。只有在这段时间过后，才会执行新的计算并生成结果。

Every time a skier loads a lift, a message is sent to the company’s service that collects data about skier traffic patterns. Using this data, the system can estimate lift wait times from the number of skiers who ride a lift and the rate they are arriving. This is an expensive calculation, taking maybe a second or more at busy times, as it requires aggregating potentially tens of thousands of lift ride records and performing the wait time calculation. For this reason, once the results are calculated, they are deemed valid for five minutes. Only after this time has elapsed is a new calculation performed and results produced.

以下代码示例显示无状态如何LiftWaitService工作。当请求到达时，服务首先检查缓存以查看是否有最新的等待时间。如果是，结果将立即返回给客户端。如果结果不在缓存中，该服务将调用下游服务，该服务执行提升等待计算并将其作为List. 然后将这些结果存储在缓存中，然后返回给客户端：

The following code example shows how a stateless LiftWaitService might work. When a request arrives, the service first checks the cache to see if the latest wait times are available. If they are, the results are immediately returned to the client. If the results are not in the cache, the service calls a downstream service which performs the lift wait calculations and returns them as a List. These results are then stored in the cache and then returned to the client:

公共类LiftWaitService {
  公共列表 getLiftWaits（字符串度假村）{
    List liftWaitTimes = cache.get(“liftwaittimes:” + Resort);
      if (liftWaitTimes == null) {
         liftWaitTimes = skiCo.getLiftWaitTimes(度假村);
         // 将结果添加到缓存中，300秒后过期
         cache.put("liftwaittimes:" + Resort, liftWaitTimes, 300);
      }
    返回电梯等待时间；
     }
   }

public class LiftWaitService {
  public List getLiftWaits(String resort) { 
    List liftWaitTimes = cache.get(“liftwaittimes:” + resort); 
      if (liftWaitTimes == null) { 
         liftWaitTimes = skiCo.getLiftWaitTimes(resort); 
         // add result to cache, expire in 300 seconds 
         cache.put("liftwaittimes:" + resort, liftWaitTimes, 300); 
      } 
    return liftWaitTimes; 
     } 
   }

缓存访问需要密钥与结果相关联。“liftwaittimes:”在此示例中，密钥由与客户端传递给服务的度假村标识符连接的字符串构成。然后，缓存对该密钥进行哈希处理，以识别缓存值所在的服务器。

Cache access requires a key with which to associate the results. In this example, the key is constructed with the string “liftwaittimes:” concatenated with the resort identifier that is passed by the client to the service. This key is then hashed by the cache to identify the server where the cached value resides.

当写入新值时缓存中，300 秒的值作为参数传递给操作put。这称为生存时间(TTL) 值。它告诉缓存，300 秒后，该键值对应从缓存中逐出，因为该值不再是当前值（也称为陈旧值）。

When a new value is written to the cache, a value of 300 seconds is passed as a parameter to the put operation. This is known as a time to live (TTL) value. It tells the cache that after 300 seconds this key-value pair should be evicted from the cache as the value is no longer current (also known as stale).

当缓存值有效时，所有请求都将使用它。这意味着无需为每个呼叫执行昂贵的电梯等待时间计算。快速网络上的缓存命中可能需要一毫秒，比提升等待时间计算快得多。当缓存值在 300 秒后被逐出时，下一个请求将导致缓存未命中。这将导致计算要存储在缓存中的新值。因此，如果我们在 5 分钟内收到N 个请求，则将从缓存中处理N -1 个请求。想象一下，如果N是 10,000？这样可以节省大量昂贵的计算，并且数据库可以使用 CPU 周期来处理其他查询。

While the cache value is valid, all requests will utilize it. This means there is no need to perform the expensive lift wait time calculation for every call. A cache hit on a fast network will take maybe a millisecond—much faster than the lift wait times calculation. When the cache value is evicted after 300 seconds, the next request will result in a cache miss. This will result in the calculation of the new values to be stored in the cache. Therefore, if we get N requests in a 5-minute period, N-1 requests are served from the cache. Imagine if N is 10,000? This is a lot of expensive calculations saved, and CPU cycles that your database can use to process other queries.

使用 TTL 等过期时间是使缓存内容无效的常见方法。它确保服务不会向客户端提供陈旧、过时的结果。它还使系统能够对缓存内容进行一些控制，而缓存内容通常是有限的。如果缓存的项目没有定期刷新，缓存就会被填满。在这种情况下，缓存将采用诸如最近最少使用或最少访问之类的策略来选择缓存条目来逐出并为更新的、及时的结果创建空间。

Using an expiry time like the TTL is a common way to invalidate cache contents. It ensures a service doesn’t deliver stale, out-of-date results to a client. It also enables the system to have some control over cache contents, which are typically limited. If cached items are not flushed periodically, the cache will fill up. In this case, a cache will adopt a policy such as least recently used or least accessed to choose cache entries to evict and create space for more current, timely results.

应用程序缓存可以显着提高吞吐量、减少延迟并提高客户端应用程序响应能力。实现这些理想品质的关键是满足缓存中尽可能多的请求。总体设计原则是最大化缓存命中率并最小化缓存未命中率。当发生缓存未命中时，必须通过查询数据库或下游服务来满足请求。然后，请求的结果可以写入缓存，从而可供进一步访问。

Application caching can provide significant throughput boosts, reduced latencies, and increased client application responsiveness. The key to achieving these desirable qualities is to satisfy as many requests as possible from the cache. The general design principle is to maximize the cache hit rate and minimize the cache miss rate. When a cache miss occurs, the request must be satisfied through querying databases or downstream services. The results of the request can then be written to the cache and hence be available for further accesses.

对于缓存命中率应该是多少，没有硬性规定，因为它取决于构建缓存内容的成本和缓存项的更新率。理想的缓存设计的读取次数远多于更新次数。这是因为当必须更新某个项目时，应用程序需要使因更新而过时的缓存条目无效。这意味着下一个请求将导致缓存未命中。¹

There’s no hard-and-fast rule on what the cache hit rate should be, as it depends on the cost of constructing the cache contents and the update rate of cached items. Ideal cache designs have many more reads than updates. This is because when an item must be updated, the application needs to invalidate cache entries that are now stale because of the update. This means the next request will result in a cache miss.¹

当项目定期更新时，缓存未命中的成本可能会抵消缓存的优势。因此，服务设计者需要仔细考虑应用程序体验的查询和更新模式，并构建产生最大效益的缓存机制。一旦服务投入生产，监控缓存使用情况也很重要，以确保命中率和未命中率符合设计预期。缓存将提供管理实用程序和 API，以监控缓存使用特征。例如，memcached 提供大量可用统计信息，包括命中和未命中计数，如下面的输出片段所示：

When items are updated regularly, the cost of cache misses can negate the benefits of the cache. Service designers therefore need to carefully consider query and update patterns an application experiences, and construct caching mechanisms that yield the most benefit. It is also crucial to monitor the cache usage once a service is in production to ensure the hit and miss rates are in line with design expectations. Caches will provide both management utilities and APIs to enable monitoring of the cache usage characteristics. For example, memcached makes a large number of statistics available, including the hit and miss counts as shown in the snippet of output below:

统计 get_hits 98567
统计 get_misss 11001
STAT 驱逐 0

STAT get_hits 98567
STAT get_misses 11001
STAT evictions 0

应用程序级缓存也称为缓存端模式。名字引用这样一个事实：如果所需的结果在缓存中可用，则应用程序代码可以有效地绕过数据存储系统。这与应用程序始终读取和写入缓存的其他缓存模式形成鲜明对比。这些被称为read-through、write-through和write-behind缓存，定义如下：

Application-level caching is also known as the cache-aside pattern. The name references the fact that the application code effectively bypasses the data storage systems if the required results are available in the cache. This contrasts with other caching patterns in which the application always reads from and writes to the cache. These are known as read-through, write-through, and write-behind caches, defined as follows:

通读

Read-through

该应用程序满足所有请求通过访问缓存。如果所需的数据在缓存中不可用，则调用加载器来访问数据系统并将结果加载到缓存中以供应用程序使用。

The application satisfies all requests by accessing the cache. If the data required is not available in the cache, a loader is invoked to access the data systems and load the results in the cache for the application to utilize.

直写式

Write-through

应用程序总是写更新到缓存。当缓存更新时，调用写入器将新的缓存值写入数据库。当数据库更新时，应用程序可以完成请求。

The application always writes updates to the cache. When the cache is updated, a writer is invoked to write the new cache values to the database. When the database is updated, the application can complete the request.

后写式

Write-behind

类似于直写式，除了应用程序不会等待将值从缓存写入数据库。如果缓存服务器在数据库更新完成之前崩溃，这会提高请求响应能力，但代价是可能会丢失更新。这也称为回写式缓存，并且是大多数数据库引擎内部使用的策略。

Like write-through, except the application does not wait for the value to be written to the database from the cache. This increases request responsiveness at the expense of possible lost updates if the cache server crashes before a database update is completed. This is also known as a write-back cache, and internally is the strategy used by most database engines.

这些缓存方法的优点在于它们简化了应用程序逻辑。应用程序始终利用缓存进行读取和写入，而缓存提供了“魔力”来确保缓存与后端存储系统正确交互。这与缓存端模式形成对比，在缓存端模式中，应用程序逻辑必须识别缓存未命中。

The beauty of these caching approaches is that they simplify application logic. Applications always utilize the cache for reads and writes, and the cache provides the “magic” to ensure the cache interacts appropriately with the backend storage systems. This contrasts with the cache-aside pattern, in which application logic must be cognizant of cache misses.

直读、直写和后写策略需要缓存技术，该技术可以通过应用程序特定的处理程序进行增强，以便在应用程序访问缓存时执行数据库读写操作。例如，NCache支持应用程序实现的提供者接口。这些会在直读式缓存的缓存未命中和直写式缓存的写入时自动调用。其他此类缓存本质上是专用数据库缓存，因此要求缓存访问与底层数据库模型相同。Amazon 的DynamoDB Accelerator (DAX)就是一个例子。DAX 位于应用程序代码和 DynamoDB 之间，透明地充当高速内存缓存，以减少数据库访问时间。

Read-through, write-through, and write-behind strategies require a cache technology that can be augmented with an application-specific handler to perform database reads and writes when the application accesses the cache. For example, NCache supports provider interfaces that the application implements. These are invoked automatically on cache misses for read-through caches and on writes for write-through caches. Other such caches are essentially dedicated database caches, and hence require cache access to be identical to the underlying database model. An example of this is Amazon’s DynamoDB Accelerator (DAX). DAX sits between the application code and DynamoDB, and transparently acts as a high-speed, in-memory cache to reduce database access times.

缓存旁路策略的一个显着优点是它能够适应缓存故障。在缓存不可用的情况下，所有请求本质上都被视为缓存未命中。性能会受到影响，但服务仍然能够满足请求。此外，由于 Redis 和memcached等缓存平台的简单、分布式哈希表模型，扩展缓存平台也很简单。由于这些原因，缓存旁路模式是大规模应用中的主要方法。可扩展的系统。

One significant advantage of the cache-aside strategy is that it is resilient to cache failure. In circumstances when the cache is unavailable, all requests are essentially handled as a cache miss. Performance will suffer, but services will still be able to satisfy requests. In addition, scaling cache-aside platforms such as Redis and mem­cached is straightforward due to their simple, distributed hash table model. For these reasons, the cache-aside pattern is the primary approach seen in massively scalable systems.

网页缓存

Web Caching

网站如此高的原因之一响应式的特点是互联网上到处都是网络缓存。Web 缓存在定义的时间段内存储给定资源（例如网页或图像）的副本。缓存拦截客户端请求，如果它们在本地缓存了所请求的资源，它们会返回副本，而不是将请求转发到目标服务。因此，可以满足许多请求而不会给服务带来负担。此外，由于缓存在物理上更接近客户端，因此请求的延迟会更低。

One of the reasons that websites are so highly responsive is that the internet is littered with web caches. Web caches store a copy of a given resource—for example, a web page or an image, for a defined time period. The caches intercept client requests and if they have a requested resource cached locally, they return the copy rather than forwarding the request to the target service. Hence, many requests can be satisfied without placing a burden on the service. Also, as the caches are physically closer to the client, the requests will have lower latencies.

图 6-2给出了 Web 缓存架构的概述。存在多个级别的缓存，从客户端的 Web 浏览器缓存和基于本地组织的缓存开始。ISP 还将实现通用 Web 代理缓存，并且可以在应用程序服务执行域内部署反向代理缓存。Web 浏览器缓存也称为私有缓存（针对单个用户）。组织和 ISP 代理缓存是支持多个用户请求的共享缓存。

Figure 6-2 gives an overview of the web caching architecture. Multiple levels of caches exist, starting with the client’s web browser cache and local organization-based caches. ISPs will also implement general web proxy caches, and reverse proxy caches can be deployed within the application services execution domain. Web browser caches are also known as private caches (for a single user). Organizational and ISP proxy caches are shared caches that support requests from multiple users.

图 6-2。互联网中的网络缓存

边缘缓存，也称为内容交付网络 (CDN)，位于全球各个战略地理位置，因此他们将经常访问的数据缓存在靠近客户端的地方。例如，视频流提供商可以在澳大利亚悉尼配置边缘缓存，为澳大利亚用户提供视频内容，而不是从位于美国的源服务器跨越太平洋流式传输内容。边缘缓存由 CDN 提供商在全球部署。Akamai 是最初的 CDN 提供商，拥有 2,000 多个地点，提供全球高达 30% 的互联网流量。对于拥有全球用户的富媒体网站来说，边缘缓存至关重要。

Edge caches, also known as content delivery networks (CDNs), live at various strategic geographical locations globally, so that they cache frequently accessed data close to clients. For example, a video streaming provider may configure an edge cache in Sydney, Australia to serve video content to Australasian users rather than streaming content across the Pacific Ocean from US-based origin servers. Edge caches are deployed globally by CDN providers. Akamai, the original CDN provider, has over 2,000 locations and delivers up to 30% of internet traffic globally. For media-rich sites with global users, edge caches are essential.

缓存通常只存储请求的结果GET，缓存键是关联的 URI GET。当客户端发送GET请求时，请求路径上可能会被一个或多个缓存拦截。任何具有所请求资源的最新副本的缓存都可以响应该请求。如果未找到缓存内容，则该请求将由服务端点（在 Web 技术术语中也称为源服务器）提供服务。

Caches typically store the results of GET requests only, and the cache key is the URI of the associated GET. When a client sends a GET request, it may be intercepted by one or more caches along the request path. Any cache with a fresh copy of the requested resource may respond to the request. If no cached content is found, the request is served by the service endpoint, which is also called the origin server in web technology parlance.

服务可以控制结果使用 HTTP 缓存指令进行缓存以及存储多长时间。服务在各种 HTTP 响应标头中设置这些指令，如以下简单示例所示：

Services can control what results are cached and for how long they are stored by using HTTP caching directives. Services set these directives in various HTTP response headers, as shown in this simple example:

回复：
HTTP/1.1 200 OK 内容长度：9842
内容类型：application/json
缓存控制：公共
日期：2019 年 3 月 26 日星期五 09:33:49 GMT
到期时间：2019 年 3 月 26 日星期五 09:38:49 GMT

Response:
HTTP/1.1 200 OK Content-Length: 9842
Content-Type: application/json 
Cache-Control: public 
Date: Fri, 26 Mar 2019 09:33:49 GMT 
Expires: Fri, 26 Mar 2019 09:38:49 GMT

我将在以下小节中描述这些指令。

I will describe these directives in the following subsections.

到期时间：2022 年 10 月 26 日星期三 09:39:00 GMT

Expires: Wed, 26 Oct 2022 09:39:00 GMT

要求：
获取 /skico.com/weather/Blackstone

回复：
HTTP/1.1 200 OK 内容长度：...
内容类型：application/json
日期：2019 年 3 月 26 日星期五 09:33:49 GMT
缓存控制：公共，最大年龄=3600
ETag：“blackstone-weather-03/26/19-v1”
<!-- 内容省略-->

Request:
GET /skico.com/weather/Blackstone

Response:
HTTP/1.1 200 OK Content-Length: ...
Content-Type: application/json 
Date: Fri, 26 Mar 2019 09:33:49 GMT 
Cache-Control: public, max-age=3600 
ETag: “blackstone-weather-03/26/19-v1"
<!-- Content omitted -->

要求：
获取 /upic.com/weather/Blackstone
如果-无-匹配：“blackstone-weather-03/26/19-v1”
回复：
HTTP/1.1 304 未修改
缓存控制：公共，最大年龄=3600

Request: 
GET /upic.com/weather/Blackstone 
If-None-Match: “blackstone-weather-03/26/19-v1"
Response:
HTTP/1.1 304 Not Modified
Cache-Control: public, max-age=3600

总结和延伸阅读

Summary and Further Reading

缓存是任何可扩展发行版的重要组成部分。缓存将许多客户端请求的信息存储在内存中，并将该信息作为客户端请求的结果。虽然信息仍然有效，但可以提供数百万次，而无需重新创建成本。

Caching is an essential component of any scalable distribution. Caching stores information that is requested by many clients in memory and serves this information as the results to client requests. While the information is still valid, it can be served potentially millions of times without the cost of re-creation.

使用分布式缓存的应用程序缓存是可扩展系统中最常见的缓存方法。此方法要求应用程序逻辑在客户端请求到达时检查缓存的值，并在可用时返回这些值。如果缓存命中率很高，大多数请求都能满足缓存结果，那么后端服务和数据库的负载就可以大大减少。

Application caching using a distributed cache is the most common approach to caching in scalable systems. This approach requires the application logic to check for cached values when a client request arrives and return these if available. If the cache hit rate is high, with most requests being satisfied with cached results, the load on backend services and databases can be considerably reduced.

互联网还具有内置的多级缓存基础设施。应用程序可以通过使用作为 HTTP 标头一部分的缓存指令来利用这一点。这些指令使服务能够指定可以缓存哪些信息、应该缓存多长时间，并采用协议来检查过时的缓存条目是否仍然有效。如果使用得当，HTTP 缓存可以显着减少下游服务和数据库的请求负载。

The internet also has a built in, multilevel caching infrastructure. Applications can exploit this through the use of cache directives that are part of HTTP headers. These directives enable a service to specify what information can be cached, for how long it should be cached, and employ a protocol for checking to see if a stale cache entry is still valid. Used wisely, HTTP caching can significantly reduce request loads on downstream services and databases.

缓存是软件和系统中一个成熟的领域，文献往往分散在许多通用和特定于产品的来源中。Gerardus Blokdyk 的Memcached，第 3 版是“万物缓存”的重要来源。（5StarCooks，2021 年）。虽然标题给出了以产品为中心的内容，但所包含的知识可以轻松地转化为具有其他竞争技术的缓存设计。

Caching is a well established area of software and systems, and the literature tends to be scattered across many generic and product-specific sources. A great source of “all things caching” is Gerardus Blokdyk’s Memcached, 3rd ed. (5StarCooks, 2021). While the title gives away the product-focused content, the knowledge contained can be translated easily to cache designs with other competing technologies.

关于 HTTP/2 的重要信息来源是Stephen Ludin 和 Javier Garza 撰写的《学习 HTTP/2：初学者实用指南》 （O'Reilly，2017 年）。尽管已经过时， Duane Wessels 的《Web Caching》（O'Reilly，2001）包含了足够的通用智慧，仍然是非常有用的参考。

A great source of information on HTTP/2 in general is Learning HTTP/2: A Practical Guide for Beginners by Stephen Ludin and Javier Garza (O’Reilly, 2017). And while dated, Web Caching by Duane Wessels (O’Reilly, 2001) contains enough generic wisdom to remain a very useful reference.

CDN 本身就是一个复杂的、特定于供应商的主题。它们适合具有丰富媒体的网站，这些网站的用户群体分布在不同的地区，需要快速的内容交付。要获得易读的 CDN 概述，Ogi Djuraskovic 的网站值得一看。

CDNs are a complex, vendor-specific topic in themselves. They come into their own for media-rich websites with a geographically dispersed group of users that require fast content delivery. For a highly readable overview of CDNs, Ogi Djuraskovic’s site is worth checking out.

消息传递简介

Introduction to Messaging

异步消息平台是一个成熟的技术领域，该领域拥有多种产品。¹久负盛名的 IBM MQ 系列出现于 1993 年，至今仍然是中流砥柱的企业系统。Java 消息服务 (JMS) 是一种 API 级规范，受到多个 JEE 供应商实现的支持。RabbitMQ（我将在本章后面用作说明）可以说是部署最广泛的开源消息传递系统。在消息传递世界中，您永远不会缺少选择。

Asynchronous messaging platforms are a mature area of technology, with multiple products in the space.¹ The venerable IBM MQ Series appeared in 1993 and is still a mainstay of enterprise systems. The Java Messaging Service (JMS), an API-level specification, is supported by multiple JEE vendor implementations. RabbitMQ, which I’ll use as an illustration later in this chapter, is arguably the most widely deployed open source messaging system. In the messaging world, you will never be short of choice.

虽然所有这些竞争产品的具体功能和 API 各不相同，但基本概念几乎相同。我将在下面的小节中介绍这些内容，然后在下一节中描述它们是如何在 RabbitMQ 中实现的。一旦您了解了一个消息传递平台的工作原理，就可以相对简单地理解竞争中固有的相似点和差异。

While the specific features and APIs vary across all these competing products, the foundational concepts are pretty much identical. I’ll cover these in the following subsections, and then describe how they are implemented in RabbitMQ in the next section. Once you appreciate how one messaging platform works, it is relatively straightforward to understand the similarities and differences inherent in the competition.

消息传递原语

Messaging Primitives

从概念上讲，消息系统包括以下内容：

Conceptually, a messaging system comprises the following:

消息队列

Message queues

存储序列的队列消息数

Queues that store a sequence of messages

制片人

Producers

发送消息至队列

Send messages to queues

消费者

Consumers

从以下位置检索消息队列

Retrieve messages from queues

消息代理

Message broker

管理一个或多个队列

Manages one or more queues

该方案如图 7-1所示。

This scheme is illustrated in Figure 7-1.

图 7-1。一个简单的消息系统

消息代理是一种管理一个或多个队列的服务。当消息从生产者发送时对于队列，代理按照消息到达的顺序将消息添加到队列中 — 基本上是一种 FIFO 方法。代理负责有效管理消息接收和保留，直到一个或多个消费者检索消息，然后将消息从队列中删除。管理多个队列和多个请求的消息代理可以有效地利用多个 vCPU 和内存来提供低延迟访问。

A message broker is a service that manages one or more queues. When messages are sent from producers to a queue, the broker adds messages to the queue in the order they arrive—basically a FIFO approach. The broker is responsible for efficiently managing message receipt and retention until one or more consumers retrieve the messages, which are then removed from the queue. Message brokers that manage many queues and many requests can effectively utilize many vCPUs and memory to provide low latency accesses.

生产者将消息发送到代理上的命名队列。许多生产者可以将消息发送到同一个队列。生产者将等待，直到收到来自代理的确认消息，然后才认为发送操作完成。

Producers send messages to a named queue on a broker. Many producers can send messages to the same queue. A producer will wait until an acknowledgment message is received from the broker before the send operation is considered complete.

许多消费者可以从同一个队列中获取消息。每条消息均由一个消费者检索。消费者检索消息有两种行为模式，称为“拉”或“推”。虽然确切的机制是特定于产品的，但基本语义在各种技术中是通用的：

Many consumers can take messages from the same queue. Each message is retrieved by exactly one consumer. There are two modes of behavior for consumers to retrieve messages, known as pull or push. While the exact mechanisms are product-specific, the basic semantics are common across technologies:

• 在拉模式（也称为轮询）中，消费者向代理发送请求，代理以下一条可供处理的消息进行响应。如果没有可用的消息，消费者必须轮询队列直到消息到达。

• In pull mode, also known as polling, consumers send a request to the broker, which responds with the next message available for processing. If there are no messages available, the consumer must poll the queue until messages arrive.

• 在推送模式下，消费者通知代理希望从队列接收消息。消费者提供了一个回调函数，当消息可用时应该调用该函数。然后，消费者会阻塞（或执行其他工作），消息代理将消息传递到回调函数，以便在消息可用时进行处理。

• In push mode, a consumer informs the broker that it wishes to receive messages from a queue. The consumer provides a callback function that should be invoked when a message is available. The consumer then blocks (or does other work) and the message broker delivers messages to the callback function for processing when they are available.

一般来说，在可用时使用推送模式会更有效，建议使用。它避免了代理可能被来自多个消费者的请求淹没，并使得在代理中更有效地实现消息传递成为可能。

Generally, utilizing the push mode when available is much more efficient and recommended. It avoids the broker being potentially swamped by requests from multiple consumers and makes it possible to implement message delivery more efficiently in the broker.

消费者还将确认收到消息。消费者确认后，代理可以自由地将消息标记为已传递并将其从队列中删除。确认可以自动或手动完成。

Consumers will also acknowledge message receipt. Upon consumer acknowledgment, the broker is free to mark a message as delivered and remove it from the queue. Acknowledgment may be done automatically or manually.

如果使用自动确认，则消息在传递给使用者后、在处理之前会立即得到确认。这提供了最低延迟的消息传递，因为可以在处理消息之前将确认发送回代理。

If automatic acknowledgment is used, messages are acknowledged as soon as they are delivered to the consumer, and before they are processed. This provides the lowest latency message delivery as the acknowledgment can be sent back to the broker before the message is processed.

通常，消费者希望确保消息在确认之前得到完全处理。在这种情况下，它将使用手动确认。这可以防止消息被传递给消费者但由于消费者崩溃而未被处理的可能性。当然，它确实会增加消息确认延迟。无论选择哪种确认模式，未确认的消息都会有效地保留在队列中，并将在稍后的某个时间传递给另一个消费者以供使用。加工。

Often a consumer will want to ensure a message is fully processed before acknowledgment. In this case, it will utilize manual acknowledgments. This guards against the possibility of a message being delivered to a consumer but not being processed due to a consumer crash. It does, of course, increase message acknowledgment latency. Regardless of the acknowledgment mode selected, unacknowledged messages effectively remain on the queue and will be delivered at some later time to another consumer for processing.

消息持久化

Message Persistence

消息代理可以管理多个同一硬件上的队列。默认情况下，消息队列通常是基于内存的，以便为生产者和消费者提供尽可能最快的服务。只要内存充足，在内存中管理队列的开销就最小。然而，如果服务器崩溃，它确实存在消息丢失的风险。

Message brokers can manage multiple queues on the same hardware. By default, message queues are typically memory based, in order to provide the fastest possible service to producers and consumers. Managing queues in memory has minimal overheads, as long as memory is plentiful. It does, however, risk message loss if the server were to crash.

为了防止消息丢失（这种做法称为数据安全），可以将队列配置为持久队列。当一条消息被由生产者放入队列中，直到消息写入磁盘后操作才完成。该方案如图 7-2所示。现在，如果消息代理发生故障，重新启动时它可以将队列内容恢复到故障之前的状态，并且不会丢失任何消息。许多应用程序无法承受丢失消息的损失，因此无法承受持久队列的损失是提供数据安全和容错所必需的。

To guard against message loss—a practice known as data safety—queues can be configured to be persistent. When a message is placed on a queue by a producer, the operation does not complete until the message is written to disk. This scheme is depicted in Figure 7-2. Now, if a message broker should fail, on reboot it can recover the queue contents to the state they existed in before the failure, and no messages will be lost. Many applications can’t afford to lose messages, and hence persistent queues are necessary to provide data safety and fault tolerance.

图 7-2。将消息持久保存到磁盘

持久队列固有地增加了发送操作的响应时间，但代价是增强了数据安全性。代理通常会在内存和磁盘上维护队列内容，以便在正常操作期间可以以最小的开销将消息传递给消费者。

Persistent queues have an inherent increase in the response time for send operations, with the trade-off being enhanced data safety. Brokers will usually maintain the queue contents in memory as well as on disk so messages can be delivered to consumers with minimal overhead during normal operations.

发布-订阅

Publish–Subscribe

消息队列将每条消息传递到恰好是一个消费者。对于许多用例来说，这正是您想要的——我的在线购买退货只需由原始供应商消耗一次——这样我就可以拿回我的钱。

Message queues deliver each message to exactly one consumer. For many use cases, this is exactly what you want—my online purchase return needs to be consumed just once by the originating vendor—so that I get my money back.

让我们扩展这个用例。假设在线零售商想要对所有购买退货进行分析，以便可以发现退货率较高的供应商并采取一些补救措施。要实现这一点，您只需将所有采购退货消息传递给相应的供应商和新的分析服务即可。这创建了一对多消息传递需求，称为发布-订阅架构模式。在发布-订阅系统中，消息队列是称为主题。主题基本上是一个消息队列，它将每条已发布的消息传递给多个订阅者之一，如图7-3所示。

Let’s extend this use case. Assume the online retailer wants to do an analysis of all purchase returns so it can detect vendors who have a high rate of returns and take some remedial action. To implement this, you could simply deliver all purchase return messages to the respective vendor and the new analysis service. This creates a one-to-many messaging requirement, which is known as a publish–subscribe architecture pattern. In publish–subscribe systems, message queues are known as topics. A topic is basically a message queue that delivers each published message to one of more subscribers, as illustrated in Figure 7-3.

图 7-3。发布-订阅代理架构

通过发布-订阅，您可以创建高度灵活和动态的系统。发布者与订阅者解耦，并且订阅者的数量可以动态变化。这使得该架构具有高度可扩展性，因为无需对现有系统进行任何更改即可添加新订阅者。它还使得多个消费者可以并行执行消息处理，从而提高性能。

With publish–subscribe, you can create highly flexible and dynamic systems. Publishers are decoupled from subscribers, and the number of subscribers can vary dynamically. This makes the architecture highly extensible as new subscribers can be added without any changes to the existing system. It also makes it possible to perform message processing by a number of consumers in parallel, thus enhancing performance.

发布-订阅给消息代理带来了额外的性能负担。经纪人有义务将每条消息传递给所有活跃订阅者。由于订阅者不可避免地会在不同时间处理和确认消息，因此代理需要保持消息可用，直到所有订阅者都使用了每条消息。利用推送模型进行消息消费为发布-订阅架构提供了最有效的解决方案。

Publish–subscribe places an additional performance burden on the message broker. The broker is obliged to deliver each message to all active subscribers. As subscribers will inevitably process and acknowledge messages at different times, the broker needs to keep messages available until all subscribers have consumed each message. Utilizing a push model for message consumption provides the most efficient solution for publish–subscribe architectures.

发布-订阅消息传递是构建分布式事件驱动架构的关键组件。在事件驱动的体系结构中，多个服务可以使用消息代理主题发布与某些状态更改相关的事件。服务可以通过订阅主题来注册对各种事件类型的兴趣。然后，针对该主题发布的每个事件都会传递给所有感兴趣的消费者服务。当第 9 章介绍微服务时，我将回到事件驱动的架构。²

Publish–subscribe messaging is a key component for building distributed, event-driven architectures. In event-driven architectures, multiple services can publish events related to some state changes using message broker topics. Services can register interest in various event types by subscribing to a topic. Each event published on the topic is then delivered to all interested consumer services. I’ll return to event-driven architectures when microservices are covered in Chapter 9.²

消息复制

Message Replication

在异步系统中，消息代理可能是一个单点故障。系统或网络故障可能会导致经纪商不可用，从而导致系统无法正常运行。这很少是一个理想的情况。

In an asynchronous system, the message broker is potentially a single point of failure. A system or network failure can cause the broker to be unavailable, making it impossible for the systems to operate normally. This is rarely a desirable situation.

因此，大多数消息代理都支持在多个代理之间物理复制逻辑队列和主题，每个代理运行在自己的节点上。如果一个代理发生故障，那么生产者和消费者可以继续使用副本之一处理消息。该架构如图 7-4所示。发布给领导者的消息会被镜像到追随者，而领导者消费的消息将从追随者中删除。

For this reason, most message brokers enable logical queues and topics to be physically replicated across multiple brokers, each running on their own node. If one broker fails, then producers and consumers can continue to process messages using one of the replicas. This architecture is illustrated in Figure 7-4. Messages published to the leader are mirrored to the follower, and messages consumed from the leader are removed from the follower.

图 7-4。消息队列复制

最常见的消息队列复制方法称为领导者-跟随者架构。指定一个代理作为领导者，生产者和消费者分别从该领导者发送和接收消息。在后台，领导者将其收到的所有消息复制（或镜像）给跟随者，并删除成功传递的消息。图7-4中显示了复制和删除操作。该方案的行为精确程度及其对经纪商绩效的影响本质上取决于实施，因此取决于产品。

The most common approach to message queue replication is known as a leader-follower architecture. One broker is designated as the leader, and producers and consumers send and receive messages respectively from this leader. In the background, the leader replicates (or mirrors) all messages it receives to the follower, and removes messages that are successfully delivered. This is shown in Figure 7-4 with the replicate and remove operations. How precisely this scheme behaves and the effects it has on broker performance is inherently implementation, and hence product dependent.

对于领导者-跟随者消息复制，跟随者被称为热备用，基本上是领导者的副本，在领导者发生故障时可用。在这种故障场景下，生产者和消费者可以通过切换到访问follower来继续操作。这也称为故障转移。故障转移是在消息代理的客户端库中实现的，因此对生产者和消费者来说是透明的。

With leader-follower message replication, the follower is known as a hot standby, basically a replica of the leader that is available if the leader fails. In such a failure scenario, producers and consumers can continue to operate by switching over to accessing the follower. This is also called failover. Failover is implemented in the client libraries for the message broker, and hence occurs transparently to producers and consumers.

实现执行队列复制的代理是一件复杂的事情。复制消息时，代理需要处理许多微妙的故障情况。当讨论转向可扩展数据管理时，我将在第 10 章和第 11章中开始提出这些问题并描述一些解决方案。

Implementing a broker that performs queue replication is a complicated affair. There are numerous subtle failure cases that the broker needs to handle when duplicating messages. I’ll start to raise these issues and describe some solutions in Chapters 10 and 11 when discussions turn to scalable data management.

警告

一些建议：不要考虑滚动自己的复制方案，或任何其他复杂的分布式算法。软件世界充满了构建特定于应用程序的分布式系统基础设施的失败尝试，只是因为可用的解决方案“不能完全满足我们的需求”或“成本太高”。相信我，您的解决方案不会像现有解决方案那样有效，而且开发成本将超出您的预期。你可能最终会扔你的代码消失了。这些算法确实很难大规模正确实施。

Some advice: don’t contemplate rolling your own replication scheme, or any other complex distributed algorithm for that matter. The software world is littered with failed attempts to build application-specific distributed systems infrastructure, just because the solutions available “don’t do it quite right for our needs” or “cost too much.” Trust me—your solution will not work as well as existing solutions and development will cost more than you could ever anticipate. You will probably end up throwing your code away. These algorithms are really hard to implement correctly at scale.

示例：RabbitMQ

Example: RabbitMQ

RabbitMQ 是分布式系统中使用最广泛的消息代理之一。你会遇到部署在所有应用领域，从金融到电信再到建筑环境控制系统。它于 2009 年左右首次发布，现已发展成为功能齐全、开源的分布式消息代理平台，支持大多数主流语言构建客户端。

RabbitMQ is one of the most widely utilized message brokers in distributed systems. You’ll encounter deployments in all application domains, from finance to telecommunications to building environment control systems. It was first released around 2009 and has developed into a full-featured, open source distributed message broker platform with support for building clients in most mainstream languages.

RabbitMQ代理已构建在 Erlang 中，主要提供对高级消息队列协议 (AMQP) 开放标准的支持。³ AMQP 作为一项合作协议定义工作诞生于金融行业。它是一种二进制协议，提供实现该协议的不同产品之间的互操作性。RabbitMQ 开箱即用，支持 AMQP v0-9-1，并通过插件支持 v1.0。

The RabbitMQ broker is built in Erlang, and primarily provides support for the Advanced Message Queuing Protocol (AMQP) open standard.³ AMQP emerged from the finance industry as a cooperative protocol definition effort. It is a binary protocol, providing interoperability between different products that implement the protocol. Out of the box, RabbitMQ supports AMQP v0-9-1, with v1.0 support via a plugin.

消息、交换和队列

Messages, Exchanges, and Queues

在 RabbitMQ 中，生产者和消费者使用客户端 API 来发送和接收来自代理的消息。经纪人提供存储转发功能消息，使用队列以 FIFO 方式处理。代理实现基于交换概念的消息传递模型，它提供了创建消息传递拓扑的灵活机制。

In RabbitMQ, producers and consumers use a client API to send and receive messages from the broker. The broker provides the store-and-forward functionality for messages, which are processed in a FIFO manner using queues. The broker implements a messaging model based on a concept called exchanges, which provide a flexible mechanism for creating messaging topologies.

交换器是一种抽象，它从生产者接收消息并将它们传递到代理中的队列。生产者只将消息写入交换器。消息包含消息有效负载和称为消息元数据的各种属性。该元数据的一个元素是路由键，它是交换器用于将消息传递到预期队列的值。

An exchange is an abstraction that receives messages from producers and delivers them to queues in the broker. Producers only ever write messages to an exchange. Messages contain a message payload and various attributes known as message metadata. One element of this metadata is the routing key, which is a value used by the exchange to deliver messages to the intended queues.

可以将交换器配置为将消息传递到一个或多个队列。消息传递算法取决于交换类型和称为绑定的规则，这些规则使用路由键在交换和队列之间建立关系。表 7-1显示了三种最常用的交换类型。

Exchanges can be configured to deliver a message to one or more queues. The message delivery algorithm depends on the exchange type and rules called bindings, which establish a relationship between an exchange and a queue using the routing key. The three most commonly used exchange types are shown in Table 7-1.

表 7-1。交易所类型

交易所类型	消息路由行为
直接的	根据与每条消息一起发布的路由键值的匹配，将消息传递到队列
话题	根据路由键和用于将队列绑定到交换器的模式的匹配，将消息传递到一个或多个队列
扇出	向所有绑定到exchange的队列投递消息，路由键被忽略

直接交换通常是用于根据匹配的路由键将每条消息传递到一个目标队列。⁴主题交换是一种基于模式匹配的更加灵活的机制，可用于实现复杂的发布-订阅消息传递拓扑。扇出交换提供了一种简单的一对多广播机制，其中每条消息都发送到所有附加队列。

Direct exchanges are typically used to deliver each message to one destination queue based on matching the routing key.⁴ Topic exchanges are a more flexible mechanism based on pattern matching that can be used to implement sophisticated publish–subscribe messaging topologies. Fanout exchanges provide a simple one-to-many broadcast mechanism, in which every message is sent to all attached queues.

图 7-5描述了直接交换的运作方式。队列与消费者的交换绑定了三个值，即“法国”、“西班牙”和“葡萄牙”。当消息从发布者到达时，交换器使用附加的路由密钥将消息传递到三个附加队列之一。

Figure 7-5 depicts how a direct exchange operates. Queues are bound to the exchange by consumers with three values, namely “France,” “Spain,” and “Portugal.” When a message arrives from a publisher, the exchange uses the attached routing key to deliver the message to one of the three attached queues.

图 7-5。RabbitMQ 直接交换的示例

以下代码摘录了如何在 Java 中配置和使用直接交换。RabbitMQ 客户端，即生产者进程和消费者进程，使用通道抽象来建立与经纪人的沟通（下一节将详细介绍渠道）。生产者在代理中创建交换，并将路由键设置为“France”的消息发布到交换。消费者在代理中创建匿名队列，将该队列绑定到发布者创建的交换器，并指定应传递使用路由键“France”发布的消息到这个队列。

The following code shows an excerpt of how a direct exchange is configured and utilized in Java. RabbitMQ clients, namely producer and consumer processes, use a channel abstraction to establish communications with the broker (more on channels in the next section). The producer creates the exchange in the broker and publishes a message to the exchange with the routing key set to “France.” A consumer creates an anonymous queue in the broker, binds the queue to the exchange created by the publisher, and specifies that messages published with the routing key “France'' should be delivered to this queue.

制作人：

Producer:

channel.exchangeDeclare(EXCHANGE_NAME, "direct");
channel.basicPublish(EXCHANGE_NAME, “法国”, null, message.getBytes());

channel.exchangeDeclare(EXCHANGE_NAME, "direct");
channel.basicPublish(EXCHANGE_NAME, “France”, null, message.getBytes());

消费者：

Consumer:

String 队列名称=channel.queueDeclare().getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, “法国”);

String queueName = channel.queueDeclare().getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, “France”);

分布与并发

Distribution and Concurrency

为了充分利用 RabbitMQ 的性能和可扩展性，您必须了解该平台是如何工作的封面。关注的问题涉及客户端和代理如何通信以及如何管理线程。

To get the most from RabbitMQ in terms of performance and scalability, you must understand how the platform works under the covers. The issues of concern relate to how clients and the broker communicate, and how threads are managed.

每个 RabbitMQ 客户端都使用 RabbitMQ 连接来连接到代理。这基本上是一个抽象基于 TCP/IP，并且可以使用用户凭据或 TLS 进行保护。创建连接是一项重量级操作，需要在客户端和服务器之间进行多次往返，因此每个客户端单个长期连接是常见的使用模式。

Each RabbitMQ client connects to a broker using a RabbitMQ connection. This is basically an abstraction on top of TCP/IP, and can be secured using user credentials or TLS. Creating connections is a heavyweight operation, requiring multiple round trips between the client and server, and hence a single long-lived connection per client is the common usage pattern.

为了发送或接收消息，客户端使用连接创建 RabbitMQ 通道。通道是客户端和代理之间的逻辑连接，仅存在于 RabbitMQ 连接的上下文中，如以下代码片段所示：

To send or receive messages, clients use the connection to create a RabbitMQ channel. Channels are a logical connection between a client and the broker, and only exist in the context of a RabbitMQ connection, as shown in the following code snippet:

ConnectionFactory connFactory = new ConnectionFactory();
连接 rmqConn = connFactory.createConnection();
频道频道 = rmqConn.createChannel();

ConnectionFactory connFactory = new ConnectionFactory();
Connection rmqConn = connFactory.createConnection();
Channel channel = rmqConn.createChannel();

可以在同一个客户端中创建多个通道来建立多个逻辑代理连接。这些通道上的所有通信都通过同一 RabbitMQ (TCP) 连接进行多路复用，如图7-6所示。创建通道需要与代理进行网络往返。因此，出于性能原因，理想情况下，通道应具有较长的生命周期，并避免通道流失，即不断创建和销毁通道。

Multiple channels can be created in the same client to establish multiple logical broker connections. All communications over these channels are multiplexed over the same RabbitMQ (TCP) connection, as shown in Figure 7-6. Creating a channel requires a network round trip to the broker. Hence for performance reasons, channels should ideally be long-lived, with channel churn, namely constantly creating and destroying channels, avoided.

图 7-6。RabbitMQ 连接和通道

提高吞吐量对于 RabbitMQ 客户端，一个常见的策略是实现多线程生产者和消费者。然而，通道不是线程安全的，这意味着每个线程都需要对通道进行独占访问。如果您的客户端具有长期存在的有状态线程并且可以为每个线程创建一个通道，则这不是问题，如图7-6所示。您启动一个线程，创建一个通道，然后发布或消耗掉。这是每线程通道模型。

To increase the throughput of RabbitMQ clients, a common strategy is to implement multithreaded producers and consumers. Channels, however, are not thread safe, meaning every thread requires exclusive access to a channel. This is not a concern if your client has long-lived, stateful threads and can create a channel per thread, as shown in Figure 7-6. You start a thread, create a channel, and publish or consume away. This is a channel-per-thread model.

然而，在 Tomcat 或 Spring 等应用服务器中，解决方案就没那么简单了。线程的生命周期和调用是由服务器平台控制的，而不是您的代码。解决方案是在服务器初始化时创建一个全局通道池。服务器线程可以按需使用这个预先创建的通道集合，而无需为每个请求创建和删除通道产生开销。每次请求到达进行处理时，服务器线程都会执行以下步骤：

In application servers such as Tomcat or Spring however, the solution is not so simple. The life cycle and invocation of threads is controlled by the server platform, not your code. The solution is to create a global channel pool upon server initialization. This precreated collection of channels can be used on demand by server threads without the overheads of channel creation and deletion per request. Each time a request arrives for processing, a server thread takes the following steps:

• 从池中检索通道

• Retrieves a channel from the pool

• 将消息发送给经纪人

• Sends the message to the broker

• 将通道返回池以供后续重用

• Returns the channel to pool for subsequent reuse

虽然没有本机 RabbitMQ 功能可以执行此操作，但在 Java 中，您可以利用Apache Commons Pool 库来实现通道池。此实现的完整代码包含在本书随附的代码存储库中。下面的代码片段显示了服务器线程如何使用Apache类borrowObject()的和方法。您可以使用 setter 方法调整此对象池的最小和最大大小，以提供应用程序所需的吞吐量：returnObject()GenericObjectPool

While there is no native RabbitMQ capability to do this, in Java you can utilize the Apache Commons Pool library to implement a channel pool. The complete code for this implementation is included in the accompanying code repository for this book. The following code snippet shows how a server thread uses the borrowObject() and returnObject() methods of the Apache GenericObjectPool class. You can tune the minimum and maximum size of this object pool using setter methods to provide the throughput your application desires:

私人布尔sendMessageToQueue（JsonObject消息）{
  尝试 {
    通道channel = pool.borrowObject();
      channel.basicPublish(//为简洁起见省略参数)
      pool.returnObject(通道);
      返回真；
    } catch (异常 e) {
      logger.info("向 RabbitMQ 发送消息失败");
      返回假；
    }
  }

private boolean sendMessageToQueue(JsonObject message) {
  try {
    Channel channel = pool.borrowObject();
      channel.basicPublish(// arguments omitted for brevity)
      pool.returnObject(channel);
      return true;
    } catch (Exception e) {
      logger.info("Failed to send message to RabbitMQ");
      return false;
    }
  }

在消费者方面，客户端创建可用于接收消息的通道。消费者可以使用 API 显式地根据需要从队列中检索消息basicGet()，如以下示例所示：

On the consumer side, clients create channels that can be used to receive messages. Consumers can explicitly retrieve messages on demand from a queue using the basicGet() API, as shown in the following example:

布尔 autoAck = true;
GetResponse 响应 =channel.basicGet(queueName, autoAck);
如果（响应==空）{
    // 没有可用的消息。决定做什么……
} 别的 {
    // 处理消息
}

boolean autoAck = true;
GetResponse response = channel.basicGet(queueName, autoAck);
if (response == null) {
    // No message available. Decide what to do …
} else {
    // process message
}

这种方法使用拉模型（轮询）。轮询效率低下，因为它涉及忙等待，迫使消费者不断地请求消息，即使没有可用的消息。在高性能系统中，这不是使用的方法。

This approach uses the pull model (polling). Polling is inefficient as it involves busy-waiting, obliging the consumer to continually ask for messages even if none are available. In high-performance systems, this is not the approach to use.

另一种更好的方法是推送模型。消费者指定为 RabbitMQ 代理向消费者发送或推送的每条消息调用的回调函数。消费者发出对 API 的调用basicConsume()。当队列中的消息可供使用者使用时，使用者上的 RabbitMQ 客户端库会调用与通道关联的另一个线程中的回调。以下代码示例演示如何使用DefaultConsumer传递basicConsume()给建立连接的类型的对象接收消息：

The alternative and preferable method is the push model. The consumer specifies a callback function that is invoked for each message the RabbitMQ broker sends, or pushes, to the consumer. Consumers issue a call to the basicConsume() API. When a message is available for the consumer from the queue, the RabbitMQ client library on the consumer invokes the callback in another thread associated with the channel. The following code example shows how to receive messages using an object of type DefaultConsumer that is passed to basicConsume() to establish a connection:

布尔 autoAck = true;
Channel.basicConsume(queueName, autoAck, "标签",
     新的默认消费者（通道）{
         @覆盖
         公共无效handleDelivery（字符串consumerTag，
                                    信封信封，
                                    AMQP.BasicProperties 属性，
                                    字节[]主体）
             抛出 IOException
         {
             // 处理消息
         }
     });

boolean autoAck = true;
channel.basicConsume(queueName, autoAck, "tag",
     new DefaultConsumer(channel) {
         @Override
         public void handleDelivery(String consumerTag,
                                    Envelope envelope,
                                    AMQP.BasicProperties properties,
                                    byte[] body)
             throws IOException
         {
             // process the message
         }
     });

接收a上的消息单通道是单线程的。这使得必须创建多个线程并分配每个线程的通道或通道池才能获得高消息消耗率。以下 Java 代码摘录展示了如何完成此操作。每个线程创建并配置自己的通道并指定回调函数threadCallback()——当传递新消息时，RabbitMQ 客户端应调用该回调函数：

Reception of messages on a single channel is single threaded. This makes it necessary to create multiple threads and allocate a channel-per-thread or channel pool in order to obtain high message consumption rates. The following Java code extract shows how this can be done. Each thread creates and configures its own channel and specifies the callback function—threadCallback()—that should be called by the RabbitMQ client when a new message is delivered:

可运行可运行 = () -> {
      尝试 {
        final Channel channel = connection.createChannel();
        Channel.queueDeclare(QUEUE_NAME, true, false, false, null);
        // 每个接收者最多一条消息
        
        final DeliverCallback threadCallback=（消费者标签，交付）
         -> {
             字符串消息 =
                 新字符串（delivery.getBody（），StandardCharsets.UTF_8）；
             // 处理消息
        };
        channel.basicConsume（QUEUE_NAME，
                             false、threadCallback、consumerTag -> {})；
        //
      } catch (IOException e) {
        logger.info(e.getMessage());
      }

Runnable runnable = () -> {
      try {
        final Channel channel = connection.createChannel();
        channel.queueDeclare(QUEUE_NAME, true, false, false, null);
        // max one message per receiver
        
        final DeliverCallback threadCallback = (consumerTag, delivery) 
         -> {
             String message = 
                 new String(delivery.getBody(), StandardCharsets.UTF_8);
             // process the message 
        };
        channel.basicConsume(QUEUE_NAME, 
                             false, threadCallback, consumerTag -> {});
        // 
      } catch (IOException e) {
        logger.info(e.getMessage());
      }

为了获得高性能和可扩展性，RabbitMQ 需要欣赏的另一个重要方面是消息代理使用的线程模型。在代理中，每个队列由单个线程管理。这意味着，如果您的队列数量至少与底层节点上的核心数量相同，则可以提高多核节点上的吞吐量。相反，如果代理节点上的利用率较高的队列多于内核数量，则可能会出现性能下降的情况。

Another important aspect of RabbitMQ to appreciate in order to obtain high performance and scalability is the thread model used by the message broker. In the broker, each queue is managed by a single thread. This means you can increase throughput on a multicore node if you have at least as many queues as cores on the underlying node. Conversely, if you have many more highly utilized queues than cores on your broker node, you are likely to see some performance degradation.

与大多数消息代理一样，当消耗率跟上生产率时，RabbitMQ 表现最佳。当队列变长，达到数万条消息时，管理队列的线程将经历更多的开销。默认情况下，代理将利用其运行的节点的 40% 的可用内存。当达到此限制时，代理将开始限制生产者，减慢代理接受消息的速度，直到内存使用率降至 40% 阈值以下。内存阈值是可配置的，并且这是一个可以设置的设置根据您的工作负载进行调整以优化消息吞吐量。⁵

Like most message brokers, RabbitMQ performs best when consumption rates keep up with production rates. When queues grow long, in the order of tens of thousands of messages, the thread managing a queue will experience more overheads. By default, the broker will utilize 40% of the available memory of the node it is running on. When this limit is reached, the broker will start to throttle producers, slowing down the rate at which the broker accepts messages, until the memory usage drops below the 40% threshold. The memory threshold is configurable and again, this is a setting that can be tuned to your workload to optimize message throughput.⁵

消息传递模式

Messaging Patterns

凭借在企业系统中的悠久使用历史，使用消息传递的应用程序存在完整的设计模式目录。虽然其中许多涉及易于构建和修改系统以及消息安全性的最佳设计实践，但其中许多直接适用于分布式系统的可扩展性。我将在接下来的部分中解释三种最常用的模式。

With a long history of usage in enterprise systems, a comprehensive catalog of design patterns exists for applications that utilize messaging. While many of these are concerned with best design practices for ease of construction and modification of systems and message security, a number apply directly to scalability in distributed systems. I’ll explain three of the most commonly utilized patterns in the next sections.

竞争的消费者

Competing Consumers

消息传递系统的常见要求就是尽快消费队列中的消息。对于竞争消费者模式，这是通过运行多个并发处理消息的消费者线程和/或进程来实现的。这使得应用程序能够根据需要水平扩展消费者来扩展消息处理。总体设计如图7-7所示。

A common requirement for messaging systems is to consume messages from a queue as quickly as possible. With the competing consumers pattern, this is achieved by running multiple consumer threads and/or processes that concurrently processes messages. This enables an application to scale out message processing by horizontally scaling the consumers as needed. The general design is shown in Figure 7-7.

图 7-7。竞争消费者模式

使用此模式，可以使用推或拉模型在消费者之间动态分发消息。使用推送方法，代理负责选择要向其传递消息的消费者。一种常见的方法是简单的循环分配算法，例如在 RabbitMQ 和 ActiveMQ 中实现的方法。这确保了消息均匀地分发给消费者。

Using this pattern, messages can be distributed across consumers dynamically using either the push or a pull model. Using the push approach, the broker is responsible for choosing a consumer to deliver a message to. A common method, which, for example, is implemented in RabbitMQ and ActiveMQ, is a simple round-robin distribution algorithm. This ensures an even distribution of messages to consumers.

通过拉取方法，消费者只需尽快处理消息即可消费消息。假设一个多线程消费者，如果一个消费者在 8 核节点上运行，另一个消费者在 2 核节点上运行，我们预计前者处理的消息量大约是后者的四倍。因此，负载平衡通过拉动方法自然发生。

With the pull approach, consumers simply consume messages as quickly as they can process them. Assuming a multithreaded consumer, if one consumer is running on an 8-core node and another on a 2-core node, we’d expect the former would process approximately four times the amount of messages of the latter. Hence, load balancing occurs naturally with the pull approach.

这种模式具有三个主要优点，即：

There are three key advantages to this pattern, namely:

可用性

Availability

如果一个消费者发生故障，系统仍然可用，并且其消息份额会简单地分发给其他竞争消费者。

If one consumer fails, the system remains available, and its share of messages is simply distributed to the other competing consumers.

故障处理

Failure handling

如果消费者失败，未确认的消息将传递给另一个队列消费者。

If a consumer fails, unacknowledged messages are delivered to another queue consumer.

动态负载平衡

Dynamic load balancing

新的消费者可以在高负载期间启动，并在负载减少时停止，而无需更改任何队列或消费者配置。

New consumers can be started under periods of high load and stopped when load is reduced, without the need to change any queue or consumer configurations.

在任何生产质量的消息传递平台中都可以找到对竞争消费者的支持。这是一种强大的扩展方式从单个队列输出消息处理。

Support for competing consumers will be found in any production-quality messaging platform. It is a powerful way to scale out message processing from a single queue.

ClientMessage msg = session.createMessage(true);
UUID idKey = UUID.randomUUID(); // 用作幂等键
msg.setStringProperty(HDR_DUPLICATE_DETECTION_ID, idKey.toString() );

ClientMessage msg = session.createMessage(true);
UUID idKey = UUID.randomUUID();  // use as idempotence key
msg.setStringProperty(HDR_DUPLICATE_DETECTION_ID, idKey.toString() );

总结和延伸阅读

Summary and Further Reading

异步消息传递是可扩展系统架构的一个组成部分。在请求经历高峰和低谷的系统中，消息传递特别有吸引力。在高峰时段，生产者可以将请求添加到队列中并快速响应客户端，而无需等待请求被处理。

Asynchronous messaging is an integral component of scalable system architectures. Messaging is particularly attractive in systems that experience peaks and troughs in request. During peak times, producers can add requests to queues and respond rapidly to clients, without having to wait for the requests to be processed.

消息传递将生产者与消费者脱钩，从而可以独立扩展它们。架构可以利用这一点，通过弹性扩展生产者和消费者来匹配流量模式并平衡消息吞吐量要求与成本。消息队列可以分布在多个代理上以扩展消息吞吐量。还可以复制队列以增强可用性。

Messaging decouples producers from consumers, making it possible to scale them independently. Architectures can take advantage of this by elastically scaling producers and consumers to match traffic patterns and balance message throughput requirements with costs. Message queues can be distributed across multiple brokers to scale message throughput. Queues can also be replicated to enhance availability.

消息传递并非没有危险。可以将重复项放置在队列中，如果队列维护在内存中，则消息可能会丢失。向消费者的交付可能会丢失，如果确认丢失，一条消息可能会被多次使用。这些数据安全问题需要在设计时注意细节，以便对重复消息和消息丢失的容忍度与系统要求相匹配。

Messaging is not without its dangers. Duplicates can be placed on queues, and messages can be lost if queues are maintained in memory. Deliveries to consumers can be lost, and a message can be consumed more than once if acknowledgments are lost. These data safety issues require attention to detail in design so that tolerance for duplicate messages and message loss is matched to the system requirements.

如果您有兴趣获得有关消息传递架构和系统的广泛而深入的知识，Gregor Hohpe 和 Bobby Woolf 所著的经典书籍《企业集成模式》（Addison-Wesley Professional，2003 年）应该是您的第一站。其他优秀的知识来源往往是特定于消息传递平台的，并且由于有很多竞争平台，因此有很多书籍可供选择。我最喜欢的关于一般消息传递智慧和 RabbitMQ 特定信息的 RabbitMQ 书籍是David Dossot 和 Lovisa Johansson 的RabbitMQ Essentials，第二版（Packt，2014 年）和Gavin M. Roy 的RabbitMQ in Depth（Manning，2017 年）。

If you are interested in acquiring a broad and deep knowledge of messaging architectures and systems, the classic book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf (Addison-Wesley Professional, 2003) should be your first stop. Other excellent sources of knowledge tend to be messaging platform specific, and as there are a lot of competing platforms, there’s a lot of books to choose from. My favorite RabbitMQ books for general messaging wisdom and RabbitMQ-specific information are RabbitMQ Essentials, 2nd ed., by David Dossot and Lovisa Johansson (Packt, 2014) and RabbitMQ in Depth by Gavin M. Roy (Manning, 2017).

最后一点，异步通信的主题以及随之而来的优点和问题将贯穿本书的其余部分。消息传递是基于微服务的架构（第 9 章）的关键组件，也是分布式数据库如何运行的基础。当我在第四部分介绍流系统和事件驱动处理时，您肯定会认识到本章的主题。

On a final note, the theme of asynchronous communications and the attendant advantages and problems will permeate the remainder of this book. Messaging is a key component of microservice-based architectures (Chapter 9) and is foundational to how distributed databases function. And you’ll certainly recognize the topics of this chapter when I cover streaming systems and event-driven processing in Part IV.

无服务器的吸引力

The Attractions of Serverless

主要组织 IT 系统从本地部署到公共云平台部署的转变似乎是不可避免的。从初创公司到政府机构再到跨国公司的组织将云视为数字化转型平台和提高业务连续性的基础技术。

The transition of major organizational IT systems from on-premises to public cloud platforms deployments seems inexorable. Organizations from startups to government agencies to multinationals see clouds as digital transformation platforms and a foundational technology to improve business continuity.

云平台的两大吸引力在于其按需付费和快速部署的能力扩大（和缩小）虚拟资源以满足波动的工作负载和数据量。当然，这种扩展能力并不是免费的。您的应用程序需要进行架构设计，以利用云平台提供的可扩展服务。当然，正如我在第 1 章中讨论的那样，成本和规模有着不可磨灭的联系。系统长时间使用的资源越多，月底的云账单就越大。

Two of the great attractions of cloud platforms are their pay-as-you-go billing and ability to rapidly scale up (and down) virtual resources to meet fluctuating workloads and data volumes. This ability to scale, of course, doesn’t come for free. Your applications need to be architected to leverage the scalable services provided by cloud platforms. And of course, as I discussed in Chapter 1, cost and scale are indelibly connected. The more resources a system utilizes for extended periods, the larger your cloud bills will be at the end of the month.

每月的云账单可以大的。真的很大。更糟糕的是，出乎意料的大！因云计算超支而出现“标价震惊”的情况比比皆是——在一项调查中，69% 的受访者经常在云预算上超支超过 25%。在一个著名的案例中，一项 Azure 任务在被注意到之前就花费了 50 万美元。造成超支的原因有很多，包括缺乏自动扩展解决方案的部署、长期容量规划不善以及云架构利用不足导致系统占用空间臃肿。

Monthly cloud bills can be big. Really big. Even worse, unexpectedly big! Cases of “sticker shock” for significant cloud overspend are rife—in one survey, 69% of respondents regularly overspent on their cloud budget by more than 25%. In one well-known case, $500K was spent on an Azure task before it was noticed. Reasons attributed to overspending are many, including lack of deployment of autoscaling solutions, poor long-term capacity planning, and inadequate exploitation of cloud architectures leading to bloated system footprints.

在云平台上，架构师面临着无数的架构决策。这些决策既广泛，涉及系统采用的整体架构模式或风格（例如微服务、N 层、事件驱动），也涉及狭义，特定于系统所基于的各个组件和云服务。

On a cloud platform, architects are confronted with a myriad of architectural decisions. These decisions are both broad, in terms of the overall architectural patterns or styles the systems adopts—for example, microservices, N-tier, event driven—and narrow, specific to individual components and the cloud services that the system is built upon.

从这个意义上说，具有架构意义的决策贯穿于云上系统设计和部署的各个方面。当您收到每月的云支出账单时，这些决定的集体后果非常明显。

In this sense, architecturally significant decisions pervade all aspects of the system design and deployment on the cloud. And the collective consequences of these decisions are highly apparent when you receive your monthly cloud spending bill.

传统上，云应用程序已部署在利用虚拟机 (VM) 的基础设施即服务 (IaaS) 平台上。在这种情况下，您需要为部署的资源付费，无论资源利用率如何。如果负载增加，弹性应用程序可以启动新的虚拟机来增加容量，通常使用云提供的负载平衡服务。您的成本基本上与您选择的虚拟机类型、部署的持续时间以及应用程序存储和传输的数据量成正比。

Traditionally, cloud applications have been deployed on an infrastructure as a service (IaaS) platform utilizing virtual machines (VMs). In this case, you pay for the resources you deploy regardless of how highly utilized they are. If load increases, elastic applications can spin up new virtual machines to increase capacity, typically using the cloud-provided load balancing service. Your costs are essentially proportional to the type of VMs you choose, the duration they are deployed for, and the amount of data the application stores and transmits.

主要云提供商提供替代方案显式配置虚拟处理资源。它们被称为无服务器平台，不需要静态配置任何计算资源。使用 AWS Lambda 或 Google App Engine (GAE) 等技术加载应用程序代码并在请求到达时按需执行。如果没有活动请求，则基本上没有资源在使用，也无需支付费用。

Major cloud providers offer an alternative to explicitly provisioning virtual processing resources. Known as serverless platforms, they do not require any compute resources to be statically provisioned. Using technologies such as AWS Lambda or Google App Engine (GAE), the application code is loaded and executed on demand, when requests arrive. If there are no active requests, there are essentially no resources in use and no charges to meet.

无服务器平台还可以管理为您自动缩放（向上和向下）。当同时请求到达时，会创建额外的处理能力来处理请求，并且在理想情况下提供一致的低响应时间。当请求负载下降时，额外的处理能力将被停用，并且不会产生任何费用。

Serverless platforms also manage autoscaling (up and down) for you. As simultaneous requests arrive, additional processing capacity is created to handle requests and, ideally, provide consistently low response times. When request loads drop, additional processing capacity is decommissioned, and no charges are incurred.

每个无服务器平台的实现细节各不相同。例如，通常支持有限数量的主流编程语言和应用服务器框架。平台提供多种配置设置，可用于平衡性能、可扩展性和成本。一般来说，成本与以下因素成正比：

Every serverless platform varies in the details of its implementation. For example, a limited number of mainstream programming languages and application server frameworks are typically supported. Platforms provide multiple configuration settings that can be used to balance performance, scalability and costs. In general, costs are proportional to the following factors:

• 选择执行请求的处理实例的类型

• The type of processing instance chosen to execute a request

• 请求数量和每个请求的处理持续时间

• The number of requests and processing duration for each request

• 每个应用程序服务器实例在无服务器基础架构上驻留多长时间

• How long each application server instance remains resident on the serverless infrastructure

然而，不同供应商所使用的具体参数差异很大。每个平台都是专有的，并且存在细微的差异。像往常一样，魔鬼潜伏在细节中。那么，让我们来探讨其中的一些专门针对 GAE 和 AWS Lambda 平台的可怕细节。

However, the exact parameters used vary considerably across vendors. Every platform is proprietary and different in subtle ways. The devil lurks, as usual, in the details. So, let’s explore some of those devilish details specifically for the GAE and AWS Lambda platforms.

谷歌应用引擎

Google App Engine

Google App Engine (GAE) 是 Google 推出的第一个产品，是现在所谓的应用程序引擎的一部分。谷歌云平台（GCP）。它有自 2011 年起全面发布，使开发人员能够在 Google 的托管云基础设施上上传和执行基于 HTTP 的应用程序服务。

Google App Engine (GAE) was the first offering from Google as part of what is now known as the Google Cloud Platform (GCP). It has been in general release since 2011 and enables developers to upload and execute HTTP-based application services on Google’s managed cloud infrastructure.

自动缩放

Autoscaling

自动缩放是您指定的选项app.yaml在您上传服务器代码时传递到 GAE 的文件中。自动缩放的应用程序由 GAE 根据默认参数值集合进行管理，您可以在app.yaml. 基本方案如图8-1所示。

Autoscaling is an option that you specify in an app.yaml file that is passed to GAE when you upload your server code. An autoscaled application is managed by GAE according to a collection of default parameter values, which you can override in your app.yaml. The basic scheme is shown in Figure 8-1.

图 8-1。GAE 自动缩放

GAE 基本上管理着根据传入流量负载为应用程序部署处理实例。如果没有传入请求，则 GAE 将不会调度任何实例。当请求到达时，GAE 会部署一个实例来处理该请求。

GAE basically manages the number of deployed processing instances for an application based on incoming traffic load. If there are no incoming requests, then GAE will not schedule any instances. When a request arrives, GAE deploys an instance to process the request.

部署一个实例可能需要几百毫秒到几秒的时间，具体取决于您所使用的编程语言。这意味着如果没有常驻实例，初始请求的延迟可能会很高。为了减轻实例加载延迟的影响，您可以指定可用于处理请求的最小实例数。当然，这需要花钱。

Deploying an instance can take anywhere between a few hundred ms to a few seconds depending on the programming language you are using. This means latency can be high for initial requests if there are no resident instances. To mitigate this instance loading latency effects, you can specify a minimum number of instances to keep available for processing requests. This, of course, costs money.

随着请求负载的增长，GAE 调度程序将动态加载更多实例来处理请求。三个参数精确控制缩放的操作方式，即：

As the request load grows, the GAE scheduler will dynamically load more instances to handle requests. Three parameters control precisely how scaling operates, namely:

目标CPU利用率

Target CPU utilization

设置CPU利用率阈值在此之上，将启动更多实例来处理流量。范围为 0.5 (50%) 至 0.95 (95%)。默认值为 0.6 (60%)。

Sets the CPU utilization threshold above which more instances will be started to handle traffic. The range is 0.5 (50%) to 0.95 (95%). The default is 0.6 (60%).

最大并发请求数

Maximum concurrent requests

设置最大并发请求数在调度程序生成新实例之前，实例可以接受。默认值为 10，最大值为 80。文档没有说明允许的最小值，但大概 1 将定义单线程服务。

Sets the maximum number of concurrent requests an instance can accept before the scheduler spawns a new instance. The default value is 10, and the maximum is 80. The documentation doesn’t state the minimum allowed value, but presumably 1 would define a single-threaded service.

目标吞吐量利用率

Target throughput utilization

这与指定的值结合使用用于指定何时启动新实例的最大并发请求数。范围为 0.5 (50%) 至 0.95 (95%)。默认值为 0.6 (60%)。它的工作原理如下：当实例的并发请求数达到等于最大并发请求值乘以目标吞吐量利用率的值时，调度程序会尝试启动新实例。

This is used in conjunction with the value specified for maximum concurrent requests to specify when a new instance is started. The range is 0.5 (50%) to 0.95 (95%). The default is 0.6 (60%). It works like this: when the number of concurrent requests for an instance reaches a value equal to maximum concurrent requests value multiplied by the target throughput utilization, the scheduler tries to start a new instance.

了解？很明显，这三个设置相互影响，使得配置有些复杂。默认情况下，在创建新实例之前，实例将处理 10 × 0.6 = 6 个并发请求。如果这 6 个（或更少）请求导致实例的 CPU 利用率超过 60%，调度程序也会尝试创建一个新实例。

Got that? As is hopefully apparent, these three settings interact with each other, making configuration somewhat complex. By default, an instance will handle 10 × 0.6 = 6 concurrent requests before a new instance is created. And if these 6 (or fewer) requests cause the CPU utilization for an instance to go over 60%, the scheduler will also try to create a new instance.

但是等等，还有更多！

But wait, there’s more!

您还可以指定值来控制 GAE 何时根据请求在请求挂起队列（参见图 8-1）中等待分派到实例进行处理的时间添加新实例。该max-pending-latency参数指定在启动其他实例来处理请求并减少延迟之前，GAE 应允许请求在待处理队列中等待的最长时间。默认值为 30 毫秒。该值越低，应用程序扩展的速度越快。而且你可能要花更多的钱。¹

You can also specify values to control when GAE adds new instances based on the time requests spend in the request pending queue (see Figure 8-1) waiting to be dispatched to an instance for processing. The max-pending-latency parameter specifies the maximum amount of time that GAE should allow a request to wait in the pending queue before starting additional instances to handle requests and reduce latency. The default value is 30 ms. The lower the value, the quicker an application will scale. And the more it will probably cost you.¹

这些自动缩放参数设置使我们能够微调服务的行为以平衡性能和成本。当然，修改这些参数将如何影响应用程序的行为取决于服务的精确功能。然而，这些参数之间存在微妙的相互作用，这一事实使得这种调整工作有些复杂。我将在本章后面的案例研究部分返回这个主题，并解释一种简单的、与平台无关的方法您可以进行服务调整。

These auto-scaling parameter settings give us the ability to fine-tune a service’s behavior to balance performance and cost. How modifying these parameters will affect an application’s behavior is, of course, dependent on the precise functionality of the service. The fact that there are subtle interplays between these parameters makes this tuning exercise somewhat complicated, however. I’ll return to this topic in the case study section later in this chapter, and explain a simple, platform-agnostic approach you can take to service tuning.

AWS Lambda

AWS Lambda

AWS Lambda 是亚马逊的无服务器平台。基本设计原则和主要特征与 GAE 和其他无服务器平台。开发人员上传代码，将其部署为称为 Lambda 函数的服务。调用时，Lambda 提供特定于语言的执行环境来运行函数代码。

AWS Lambda is Amazon’s serverless platform. The underlying design principles and major features echo that of GAE and other serverless platforms. Developers upload code which is deployed as services known as Lambda functions. When invoked, Lambda supplies a language-specific execution environment to run the function code.

以下代码显示了 Python Lambda 函数的简单示例。该函数只是从输入事件中提取一条消息，并将其作为 HTTP200响应的一部分原封不动地返回。通常，您实现一个接受事件和上下文参数的函数。该事件是一个 JSON 格式的文档，封装了供 Lambda 函数处理的数据。例如，如果 Lambda 函数处理 HTTP 请求，则事件将包含 HTTP 标头和请求正文。上下文包含有关函数和运行时环境的元数据，例如函数版本号和执行环境中的可用内存：

A simple example of a Python Lambda function is shown in the following code. This function simply extracts a message from the input event and returns it unaltered as part of an HTTP 200 response. In general, you implement a function that takes an event and a context parameter. The event is a JSON-formatted document encapsulating data for a Lambda function to process. For example, if the Lambda function handles HTTP requests, the event will contain HTTP headers and the request body. The context contains metadata about the function and runtime environment, such as the function version number and available memory in the execution environment:

导入 json

def lambda_handler（事件，上下文）：
     event_body = json.loads(事件['body'])
     响应={
        “状态代码”：200，
        'body': json.dumps({ event_body['message'] })
    }

    返回响应

import json

def lambda_handler(event, context):
     event_body = json.loads(event[‘body’])
     response = {
        'statusCode': 200,
        'body': json.dumps({ event_body[‘message’] })
    }

    return response

外部客户端可以通过 HTTP 调用 Lambda 函数。它们还可以与其他 AWS 服务紧密集成。例如，这使得 Lambda 函数能够在新数据写入 AWS S3 存储服务或监控事件发送到 AWS CloudWatch 服务时动态触发。如果您的应用程序深深嵌入到 AWS 生态系统中，则 Lambda 函数在设计和部署架构时非常有用。

Lambda functions can be invoked by external clients over HTTP. They can also be tightly integrated with other AWS services. For example, this enables Lambda functions to be dynamically triggered when new data is written to the AWS S3 storage service or a monitoring event is sent to the AWS CloudWatch service. If your application is deeply embedded in the AWS ecosystem, Lambda functions can be of great utility in designing and deploying your architecture.

鉴于无服务器平台之间的核心相似之处，在本节中，我将仅从可扩展性和成本角度关注 Lambda 的差异化功能。

Given the core similarities between serverless platforms, in this section I’ll just focus on the differentiating features of Lambda from a scalability and cost perspective.

可扩展性

Scalability

随着并发请求数对于功能的增加，Lambda 将部署更多运行时实例来扩展处理。如果请求负载继续增长，Lambda 会重用可用实例并根据需要创建新实例。最终，当请求负载下降时，Lambda 通过停止未使用的实例来缩小规模。无论如何，这是简单的版本。事实上，情况要复杂一些。

As the number of concurrent requests for a function increases, Lambda will deploy more runtime instances to scale the processing. If the request load continues to grow, Lambda reuses available instances and creates new instances as needed. Eventually, when the request load falls, Lambda scales down by stopping unused instances. That’s the simple version, anyway. In reality, it is a tad more complicated.

所有 Lambda 函数都具有内置的请求突发并发限制。有趣的是，这个默认爆了限制因部署该函数的 AWS 区域而异。例如，在美国西部（俄勒冈），一个函数可以扩展到 3,000 个实例来处理突发请求，而在欧洲（法兰克福），该限制为 1,000 个实例。⁵

All Lambda functions have a built-in concurrency limit for request bursts. Interestingly, this default burst limit varies depending on the AWS region where the function is deployed. For example, in US West (Oregon), a function can scale up to 3,000 instances to handle a burst of requests, whereas in Europe (Frankfurt) the limit is 1,000 instances.⁵

无论在哪个区域，一旦达到突发限制，函数就可以以每分钟 500 个实例的速度扩展。这种情况一直持续到需求得到满足并且请求开始下降为止。如果请求负载超过每分钟可处理 500 个额外实例的容量，Lambda 将限制该函数并向客户端返回 HTTP 429，客户端必须重试该请求。

Regardless of the region, once the burst limit is reached, a function can scale at a rate of 500 instances per minute. This continues until the demand is satisfied and requests start to drop off. If the request load exceeds the capacity that can be processed by 500 additional instances per minute, Lambda throttles the function and returns an HTTP 429 to clients, who must retry the request.

这种行为如图 8-2所示。在请求突发期间，实例数量快速增长，直至达到区域定义的突发限制。此后，每分钟只能部署 500 个新实例。在此期间，可用实例无法满足的请求将受到限制。随着请求负载下降，实例将从平台中删除，直到流量恢复稳定状态。

This behavior is depicted in Figure 8-2. During the request burst, the number of instances grows rapidly up to the region-defined burst limit. After that, only 500 new instances can be deployed per minute. During this time, requests that cannot be satisfied by the available instances are throttled. As the request load drops, instances are removed from the platform until a steady state of traffic resumes.

函数可以处理的确切并发客户端请求数量取决于该函数的处理时间。例如，假设我们部署了 3,000 个实例，每个请求平均需要 100 毫秒来处理。这意味着每个实例每秒可以处理 10 个请求，每秒的最大吞吐量为 (3,000 × 10) = 30,000 个请求。

Precisely how many concurrent client requests a function can handle depends on the processing time for the function. For example, assume we have 3,000 deployed instances, and each request takes on average 100 milliseconds to process. This means that each instance can process 10 requests per second, giving a maximum throughput of (3,000 × 10) = 30,000 requests per second.

图 8-2。扩展 AWS Lambda 函数

为了完整了解这一情况，您需要注意突发并发限制实际上适用于与单个 AWS 账户关联的区域中的所有功能。因此，如果您在同一区域的一个账户下部署三个不同的 Lambda 函数，则它们部署的实例总数由决定扩展行为的突发限制控制。这意味着，如果一个功能突然意外地重载，它可能会消耗突发限制，并对希望同时扩展的其他功能的可用性产生负面影响。

To complete the picture, you need to be aware that the burst concurrency limit actually applies to all functions in the region associated with a single AWS account. So, if you deploy three different Lambda functions in the same region under one account, their collective number of deployed instances is controlled by the burst limit that determines the scaling behavior. This means if one function is suddenly and unexpectedly heavily loaded, it can consume the burst limit and negatively impact the availability of other functions that wish to scale at the same time.

为了解决这种潜在冲突，您可以微调与同一区域的同一 AWS 账户下部署的每个单独 Lambda 函数关联的并发级别。⁶这称为保留并发。每个单独的功能都可以与小于突发限制的值相关联。⁷该值定义可以同时执行的该函数的最大实例数。

To address this potential conflict, you can fine-tune the concurrency levels associated with each individual Lambda function deployed under the same AWS account in the same region.⁶ This is known as reserved concurrency. Each individual function can be associated with a value that is less than the burst limit.⁷ This value defines the maximum number of instances of that function that can be executed concurrently.

预留并发有两个含义：

Reserved concurrency has two implications:

• 具有保留并发性的 Lambda 函数始终具有专用于其自身调用的执行能力。它不会因该区域中其他函数的并发调用而意外地饥饿。

• The Lambda function with reserved concurrency always has execution capacity available exclusively for its own invocations. It cannot be unexpectedly starved by concurrent invocations of other functions in the region.

• 保留容量限制了该函数的最大驻留实例数。当实例数量达到保留值时无法处理的请求将失败并出现 HTTP 429 错误。

• The reserved capacity caps the maximum number of resident instances for that function. Requests that cannot be processed when the number of instances is at the reserved value fail with an HTTP 429 error.

从本次讨论中可以明显看出，AWS Lambda 提供了强大而灵活的无服务器环境。只要小心，运行时环境就可以配置为有效扩展，以处理大容量、突发的请求负载。它已成为许多组织内部和面向客户的应用程序的 AWS 工具箱不可或缺的一部分。⁸

As should be apparent from this discussion, AWS Lambda provides a powerful and flexible serverless environment. With care, the runtime environment can be configured to scale effectively to handle high-volume, bursty request loads. It has become an integral part of the AWS toolbox for many organizations’ internal and customer-facing applications.⁸

案例研究：平衡吞吐量和成本

Case Study: Balancing Throughput and Costs

从无服务器平台以最低成本获得所需的性能和可扩展性几乎总是需要调整运行时参数设置。当您的申请是每天可能处理数百万个请求，即使成本降低 10% 也可以节省大量资金。当然，足以让你的老板和客户满意。

Getting the required performance and scalability at lowest cost from a serverless platform almost always requires tweaking of the runtime parameter settings. When your application is potentially processing many millions of requests per day, even a 10% cost reduction can result in significant monetary savings. Certainly, enough to make your boss and clients happy.

所有无服务器平台的可调整参数设置均有所不同。有些相对简单，例如 AWS Lambda，其中选择函数的内存量是主要的调整参数。另一个极端可能是 Azure Functions，它具有多个参数设置和部署限制，这些设置和部署限制根据选择的三个托管计划中的哪一个而有所不同。⁹

All serverless platforms vary in the parameter settings you can tune. Some are relatively straightforward, such as AWS Lambda in which choosing the amount of memory for a function is the dominant tuning parameter. The other extreme is perhaps Azure Functions, which has multiple parameter settings and deployment limits that differ based on which of three hosting plans are selected.⁹

GAE 位于这两者之间，具有一些控制自动缩放行为的参数。我将以此作为如何进行应用程序调优的示例。

GAE sits between these two, with a handful of parameters that govern autoscaling behavior. I’ll use this as an example of how to approach application tuning.

总结和延伸阅读

Summary and Further Reading

无服务器平台是构建可扩展应用程序的强大工具。它们消除了与管理和更新显式分配的虚拟机集群相关的许多部署复杂性。部署就像开发服务代码并将其与配置文件一起上传到平台一样简单。您使用的无服务器平台会负责剩下的工作。

Serverless platforms are a powerful tool for building scalable applications. They eliminate many of the deployment complexities associated with managing and updating clusters of explicitly allocated virtual machines. Deployment is as simple as developing the service’s code, and uploading it to the platform along with a configuration file. The serverless platform you are using takes care of the rest.

无论如何，理论上来说。

In theory, anyway.

当然，在实践中，您可以使用一些重要的旋钮和旋钮来调整底层无服务器平台管理功能的方式。这些都是特定于平台的，但许多都与性能和可扩展性以及最终您支付的金额有关。本章中的案例研究说明了这种关系，并为您提供了一种方法，您可以利用该方法找到难以捉摸的最佳点，以比默认平台参数设置更低的成本提供所需的性能。

In practice, of course, there are important dials and knobs that you can use to tune the way the underlying serverless platforms manage your functions. These are all platform-specific, but many relate to performance and scalability, and ultimately the amount of money you pay. The case study in this chapter illustrated this relationship and provided you with an approach you can utilize to find that elusive sweet spot that provides the required performance at lower costs than the default platform parameter settings provide.

要充分利用无服务器计算的优势，您需要购买云服务提供商的资金。有很多可供选择，但如果您决定迁移到新平台，所有这些都会伴随着供应商锁定和下游痛苦和痛苦。

Exploiting the benefits of serverless computing requires you to buy into a cloud service provider. There are many to choose from, but all come with the attendant vendor lock-in and downstream pain and suffering if you ever decide to migrate to a new platform.

有一些开源无服务器平台，例如Apache OpenWhisk，可以部署到本地硬件或云配置的虚拟资源。还有一些独立于提供商的解决方案，例如无服务器框架。这些使得将无服务器编写的应用程序部署到许多主流云提供商（包括所有常见的云提供商）成为可能。这提供了代码可移植性，但并不能使系统免受不同提供商部署环境的复杂性的影响。不可避免的是，在新平台上实现所需的性能、可扩展性和安全性并不是一件容易的事。

There are open source serverless platforms such as Apache OpenWhisk that can be deployed to on-premises hardware or cloud-provisioned virtual resources. There are also solutions such as the Serverless Framework that are provider-independent. These make it possible to deploy applications written in Serverless to a number of mainstream cloud providers, including all the usual suspects. This delivers code portability but does not insulate the system from the complexities of different provider deployment environments. Inevitably, achieving the required performance, scalability, and security on a new platform is not going to be a walk in the park.

Jason Katzer 的Learning Serverless（O'Reilly，2020）是有关无服务器计算的重要信息来源。我还推荐两篇非常有趣的文章，讨论无服务器计算的当前技术水平和未来的可能性。这些都是：

A great source of information on serverless computing is Jason Katzer’s Learning Serverless (O’Reilly, 2020). I’d also recommend two extremely interesting articles that discuss the current state of the art and future possibilities for serverless computing. These are:

• D. Taibi 等人，“无服务器计算：我们现在在哪里，我们将走向何方？” IEEE 软件。38，没有。1（2021 年 1 月至 2 月）：25-31，doi：10.1109/MS.2020.3028708。

• D. Taibi et al., “Serverless Computing: Where Are We Now, and Where Are We Heading?” IEEE Software. 38, no. 1 (Jan.–Feb. 2021): 25–31, doi: 10.1109/MS.2020.3028708.

• J. Schleier-Smith 等人，“无服务器计算是什么以及应该成为什么：云计算的下一阶段”，ACM 64 通讯，第 1 期。5（2021 年 5 月）：76–84。

• J. Schleier-Smith et al., “What Serverless Computing Is and Should Become: The Next Phase of Cloud Computing,” Communications of the ACM 64, no. 5 (May 2021): 76–84.

最后，无服务器平台是实现微服务架构的常用技术。微服务是一种架构模式，用于将应用程序分解为多个独立可部署和可扩展的部分。这种设计方法非常适合基于无服务器的实现，并且方便，这是我们在下一章中讨论的主题。

Finally, serverless platforms are a common technology for implementing microservices architectures. Microservices are an architectural pattern for decomposing an application into multiple independently deployable and scalable parts. This design approach is highly amenable to a serverless-based implementation, and conveniently, is the topic we cover in the next chapter.

微服务运动

The Movement to Microservices

在许多方面，基于微服务的架构都受益于过去十年中出现的软件工程和技术创新的融合。敏捷的小型团队、持续开发和集成实践以及部署技术共同为微服务所体现的细粒度架构方法提供了肥沃的土壤。基于微服务的架构是利用这些进步来部署灵活、可扩展和可伸缩系统的催化剂。让我们来看看它们的起源和一些特征。

In many ways, microservice-based architectures have benefited from a confluence of software engineering and technology innovation that has emerged over the last decade. Small, agile teams, continuous development and integration practices, and deployment technologies have collectively provided fertile ground for the fine-grained architectural approach embodied by microservices. Microservice-based architectures are a catalyst for exploiting these advances to deploy flexible, extensible, and scalable systems. Let’s examine their origins and some features.

整体应用程序

Monolithic Applications

自 IT 系统出现以来，整体架构风格主导了企业应用程序。本质上，这种风格将应用程序分解为多个逻辑模块或服务，这些模块或服务作为单个应用程序构建和部署。这些服务提供可由外部客户端调用的端点。端点提供安全性和输入验证，然后将请求委托给共享业务逻辑，而共享业务逻辑又将通过数据访问对象（DAO）层。图 9-1中描述了一个示例大学管理系统的设计，该系统具有处理学生课程作业和时间表、房间安排、费用支付以及教师和顾问交互的功能

Since the dawn of IT systems, the monolithic architectural style has dominated enterprise applications. Essentially, this style decomposes an application into multiple logical modules or services, which are built and deployed as a single application. These services offer endpoints that can be called by external clients. Endpoints provide security and input validation and then delegate the requests to shared business logic, which in turn will access a persistent store through a data access objects (DAO) layer. This design is depicted in Figure 9-1 for an example university management system that has capabilities to handle student course assignments and timetables, room scheduling, fee payments, and faculty and advisor interactions

这种架构鼓励创建可在服务实现之间共享的可重用业务逻辑和 DAO。DAO 映射到数据库实体，所有服务实现共享一个数据库。

This architecture encourages the creation of reusable business logic and DAOs that can be shared across service implementations. DAOs are mapped to database entities, and all service implementations share a single database.

IBM WebSphere 和 Microsoft .NET 等流行平台支持将所有服务构建和部署为单个可执行包。这就是术语“单体”（完整的应用程序）的由来。API、业务逻辑、数据访问等等都包含在单个部署工件中。

Popular platforms such as IBM WebSphere and Microsoft .NET enable all the services to be built and deployed as a single executable package. This is where the term monolith—the complete application—originates. APIs, business logic, data access, and so forth are all wrapped up in a single deployment artifact.

图 9-1。整体应用程序示例

考虑到该方法的寿命长，单体应用程序具有许多优点，这并不奇怪。该架构方法很好理解，为新应用程序提供了坚实的基础。它在多种语言的开发框架中享有广泛的自动化。测试很简单，部署也很简单，因为只需管理一个应用程序包。由于应用程序在一台（可能非常强大）服务器上运行，因此系统和错误监控也得到了简化。

Monolithic applications, unsurprisingly given the longevity of the approach, have many advantages. The architectural approach is well understood and provides a solid foundation for new applications. It enjoys extensive automation in development frameworks in many languages. Testing is straightforward, as is deployment as there is just a single application package to manage. System and error monitoring is also simplified as the application runs on one (probably quite powerful) server.

扩大规模是最简单的方法提高整体应用程序的响应能力和容量。横向扩展也是可能的。可以配置两个或多个整体副本，并使用负载平衡器来分发请求。只要负载均衡器支持有状态设计的会话关联性，这适用于有状态和无状态服务。

Scaling up is the simplest way to improve responsiveness and capacity for monolithic applications. Scaling out is also possible. Two or more copies of the monolith can be provisioned, and a load balancer utilized to distribute requests. This works for both stateful and stateless services, as long as the load balancer supports session affinity for stateful designs.

随着系统功能和请求量的增长，单体应用可能会开始出现问题。这个问题有两个基本要素：

Monoliths can start to become problematic as system features and request volumes grow. This problem has two fundamental elements:

代码库复杂性

Code base complexity

随着应用程序的大小和工程团队不断壮大，添加新功能、测试和重构变得越来越困难。技术债务不可避免地会增加，如果没有在工程方面进行大量投资，代码就会变得越来越脆弱。如果没有持续一致的重构努力来维护架构完整性和代码质量，工程就会变得更加困难。推出新功能的开发节奏加快。

As the size of the application and engineering team grows, adding new features, testing, and refactoring become progressively more difficult. Technical debt inevitably builds, and without significant investments in engineering, the code becomes more and more fragile. Engineering becomes harder without continual and concerted refactoring efforts to maintain architectural integrity and code quality. Development cadence increases for rolling out new features.

横向扩展

Scaling out

您可以通过以下方式进行扩展在多个节点上复制应用程序以增加容量。但这意味着每次都要复制整个应用程序（整体）。AdvisorChat在大学管理系统中，假设向学生发布对移动设备的支持后，服务的使用量突然激增。您可以部署新的副本来处理聊天消息量，但新节点需要足够强大且数量充足才能运行完整的应用程序。您无法轻松地提取聊天服务功能并独立扩展它。

You can scale out by replicating the application on multiple nodes to add capacity. But this means replicating the entire application (the monolith) every time. In the university management system, assume a sudden spike in the use of the AdvisorChat service occurs as support for mobile devices is released to the students. You can deploy new replicas to handle the chat message volume, but the new nodes need to be powerful and numerous enough to run the complete application. You can’t easily just pull out the chat service functionality and scale it independently.

这就是微服务登场的地方。他们提供工程解决方案并扩展几乎是巨大的挑战随着请求量的快速增长，不可避免地要面临。

This is where microservices enter the scene. They provide solutions to engineering and scale out challenges that monoliths almost inevitably face as the volume of requests grows rapidly.

打破整体

Breaking Up the Monolith

微服务架构分解了将应用程序功能分解为多个独立的服务，这些服务在必要时进行通信和协调。图 9-2显示了如何使用微服务设计图 9-1中的大学管理系统。每个微服务都是完全独立的，在需要时封装自己的数据存储，并提供用于通信的 API。

A microservice architecture decomposes the application functionality into multiple independent services that communicate and coordinate when necessary. Figure 9-2 shows how the university management system from Figure 9-1 might be designed using microservices. Each microservice is totally self-contained, encapsulating its own data storage where needed, and offers an API for communications.

图 9-2。微服务架构示例

随着系统代码大小和请求负载的增长，微服务具有以下优势：

Microservices offer the following advantages as systems grow in code size and request load:

代码库

Code base

遵循两个比萨饼的规则，个人服务的复杂程度不应超过小型团队可以构建、发展和管理的程度。由于微服务是一个黑匣子，团队有充分的自主权选择自己的开发堆栈和数据管理平台。¹鉴于精心设计的微服务支持的功能范围更窄、高度内聚，这应该会降低代码复杂性并提高新功能的开发节奏。此外，微服务的修订版可以根据需要独立部署。如果微服务支持的 API 稳定，则更改对于依赖的服务是透明的。

Following the two-pizza rule, an individual service should not be more complex than a small team size can build, evolve, and manage. As a microservice is a black box, the team has full autonomy to choose their own development stack and data management platform.¹ Given the narrower, highly cohesive scope of functionality that a well-designed microservice supports, this should result in lower code complexity and higher development cadence for new features. In addition, revisions of the microservice can be independently deployed as needed. If the API the microservice supports is stable, the change is transparent to dependent services.

向外扩展

Scale out

单个微服务可以扩展以满足请求量和延迟要求。例如，为了满足不断要求和聊天的学生，AdvisorChat可以根据需要在其自己的负载均衡器后面复制微服务，以提供较短的响应时间。如图 9-3所示。其他轻负载的服务可以简单地在单个节点上运行或以低成本进行复制，以消除单点故障并增强可用性。

Individual microservices can be scaled out to meet request volume and latency requirements. For example, to satisfy the ever-demanding and chatting students, the AdvisorChat microservice can be replicated as needed behind its own load balancer to provide low response times. This is depicted in Figure 9-3. Other services that experience light loads can simply run on a single node or be replicated at low cost to eliminate single points of failure and enhance availability.

图 9-3。独立扩展微服务

迁移到微服务架构时的关键设计决策之一是如何将系统功能分解为单独的服务。领域驱动设计 (DDD)提供了一种识别微服务的合适方法，因为微服务必然具有独立性，很好地映射到 DDD 中的有界上下文概念。这些主题超出了本章的范围，但对于基于微服务的应用程序架构师来说是必不可少的知识。

One of the key design decisions when moving to a microservices architecture is how to decompose the system functionality into individual services. Domain-driven design (DDD) provides a suitable method for identifying microservices, as the necessarily self-contained nature of microservices maps well to the notion of bounded contexts in DDD. These topics are beyond the scope of this chapter but are essential knowledge for architects of microservice-based applications.

但总有一个平衡的行为。微服务本质上是分布式的。通常，需要对域模型的纯度进行分析和调整，以满足分布式通信成本以及系统管理和监控复杂性的现实情况。您需要考虑请求负载以及服务这些请求所需的交互，以便微服务之间的多次交互不会导致过多的延迟。

There is always a balancing act though. Microservices are by their very nature distributed. Often, the purity of the domain model needs to be analyzed and adjusted to meet the reality of the costs of distributed communications and the complexity of system management and monitoring. You need to factor in request loads and the interactions needed to serve these requests, so that excessive latencies aren’t incurred by multiple interactions between microservices.

例如，Faculty和Funding是微服务的优秀候选者。然而，如果满足“获得教师资助”或“为教师寻找资助机会”等请求而导致过多的沟通，那么性能和可靠性可能会受到影响。在这种情况下，合并微服务可能是一个明智的选择。另一种常见的方法是跨耦合微服务复制数据。这使得服务能够在本地访问其所需的数据，从而简化设计并减少数据访问响应时间。

For example, Faculty and Funding are excellent candidates for microservices. However, if satisfying requests such as “get funding by faculty” or “find funding opportunities for faculty” incur excessive communications, performance and reliability could be impacted. Merging microservices may be a sensible option in such circumstances. Another common approach is to duplicate data across coupled microservices. This enables a service to access the data it needs locally, simplifying the design and reducing data access response times.

当然，重复数据是一种权衡。需要额外的存储容量和开发工作来确保所有重复的数据收敛到一致的状态。当数据发生变化时可以立即发起重复数据更新，试图最小化重复数据不一致的时间间隔。或者，如果业务上下文允许，则可以进行定期复制（例如，每小时或每天），可能由在请求负载较低时调用的计划任务来执行。随着应用程序对性能和可扩展性的需求不断增长，与重复数据所带来的问题相比，重复数据的成本和复杂性通常很小。将出现系统的重大重构。

Duplicate data is, of course, a trade-off. It takes additional storage capacity and development effort to ensure all duplicated data converges to a consistent state. Duplicate data updates can be initiated immediately when data changes to attempt to minimize the time interval that the duplicates are inconsistent. Alternatively, if the business context allows, periodic duplication (e.g., hourly or daily) can operate, perhaps executed by a scheduled task that is invoked when request loads are low. As the demands on performance and scalability on an application grow, the cost and complexity of duplicate data is typically small compared to the problems that a major refactoring of the system would present.

部署微服务

Deploying Microservices

支持频繁更新并受益由于小型团队提供的敏捷性，您需要能够轻松快速地部署新的微服务版本。这就是我们开始侵犯持续部署和 DevOps 世界的地方，这远远超出了本书的范围（有关阅读建议，请参阅“摘要和进一步阅读” ）。尽管如此，部署选项还是会影响微服务的可扩展性。我将在本节中仅描述一种部署微服务的常见方法。

To support frequent updates and benefit from the agility afforded by small teams, you need to be able to deploy new microservice versions easily and quickly. This is where we start to infringe on the world of continuous deployment and DevOps, which is way beyond the scope of this book (see “Summary and Further Reading” for reading recommendations). Still, deployment options impinge on the ease of scalability for a microservice. I’ll just describe one common approach for deploying microservices in this section.

正如我在第 8 章中所描述的，无服务器处理平台是一种有吸引力的微服务部署方法。可以构建微服务以在您选择的无服务器平台上公开其 API。无服务器选项具有三个优点：

Serverless processing platforms, as I described in Chapter 8, are an attractive microservices deployment approach. A microservice can be built to expose its API on the serverless platform of your choice. The serverless option has three advantages:

部署简单

Deployment is simple

只需将微服务的新可执行包上传到您为函数配置的端点即可。

Just upload the new executable package for your microservice to the endpoint you have configured for your function.

按用量付费

Pay by usage

如果您的服务存在请求量较低的时期，您的成本就会很低，甚至为零。

If your service has periods of low-volume requests, your costs are low, even zero.

易于扩展

Ease of scaling

您选择的平台负责处理功能的扩展。您可以通过配置参数精确控制其工作方式，但无服务器选项承担了可扩展性的重担。

The platform you choose handles scaling of your function. You control precisely how this works through configuration parameters, but the serverless option takes the heavy lifting out of scalability.

当您在无服务器平台上部署所有微服务时，您会公开客户端需要调用的多个端点。这会带来复杂性，因为客户端需要能够发现每个微服务的位置（主机 IP 地址和端口）。如果您决定通过组合两个微服务来重构您的微服务，或者为了消除网络调用，该怎么办？或者将 API 实现从一个微服务迁移到另一个微服务？或者甚至更改 API 的端点（IP 地址和端口）？

When you deploy all your microservices on a serverless platform, you expose multiple endpoints that clients need to invoke. This introduces complexity as clients need to be able to discover the location (host IP address and port) of each microservice. What if you decide to refactor your microservices by perhaps combining two in or order to eliminate network calls? Or move an API implementation from one microservice to another? Or even change the endpoint (IP address and port) of an API?

直接向客户端公开后端更改从来都不是一个好主意。《四人帮》一书多年前就通过面向对象系统中的外观模式教会了我们这一点。²在微服务中，您可以使用API 网关模式来利用类似的方法。API 网关本质上充当所有客户端请求的单个入口点，如图9-4所示。它将客户端与实现应用程序功能的微服务的底层架构隔离开来。现在，如果您重构底层 API，甚至选择部署在完全不同的平台（例如私有云）上，客户就不会注意到更改。

Exposing backend changes directly to clients is never a good idea. The Gang of Four book taught us this many years ago with the façade pattern in object-oriented systems.² In microservices, you can exploit an analogous approach using the API gateway pattern. An API gateway essentially acts as a single entry point for all client requests, as shown in Figure 9-4. It insulates clients from the underlying architecture of the microservices that implement the application functionality. Now, if you refactor your underlying APIs or even choose to deploy on a radically different platform such as a private cloud, clients are oblivious to changes.

图 9-4。API网关模式

您可以在系统中利用多种 API 网关实现。这些范围从强大的开源解决方案（例如NGINX Plus和Kong API 网关）到云供应商特定的托管产品。如下列出的一般功能范围是相似的：

There are multiple API gateway implementations you can exploit in your systems. These range from powerful open source solutions such as the NGINX Plus and Kong API gateways to cloud vendor–specific managed offerings. The general range of functions, listed as follows, is similar:

• 以低毫秒延迟将传入的客户端 API 请求代理到实现 API 的后端微服务。由 API 网关处理的面向客户端的 API 与后端微服务 API 之间的映射是通过管理工具或配置文件执行的。不同产品的功能和性能存在差异，有时差异很大，尤其是在高请求​​负载下。³

• Proxy incoming client API requests with low millisecond latencies to backend microservices that implement the API. Mapping between client-facing APIs, handled by the API gateway, and backend microservice APIs is performed through admin tools or configuration files. Capabilities and performance vary across products, sometimes quite significantly, especially under high request loads.³

• 为请求提供身份验证和授权。

• Provide authentication and authorization for requests.

• 定义限制每个 API 的规则。设置微服务每秒可以处理的最大请求数可用于确保后端处理不会不堪重负。

• Define rules for throttling each API. Setting the maximum number of requests a microservice can handle per second can be used to ensure backend processing is not overwhelmed.

• 支持API结果缓存，以便在不调用后端服务的情况下处理请求。

• Support a cache for API results so that requests can be handled without invoking backend services.

• 与监控工具集成以支持 API 使用情况、延迟和错误指标的分析。

• Integrate with monitoring tools to support analysis of API usage, latencies, and error metrics.

当然，在请求激增的情况下，API 网关存在成为瓶颈的危险。API 网关的处理方式因产品而异。例如，AWS API Gateway 具有每秒 10K 请求的限制，以及高达 5K 请求/秒的额外突发配额。⁴ Kong API网关是无状态的，因此可以部署多个实例并使用负载平衡器。

Under heavy request spikes, there is, of course, the danger of the API gateway becoming a bottleneck. How this is handled by your API gateway is product-specific. For example, AWS API Gateway has a 10K requests per second limit, with an additional burst quota of up to 5K requests/second.⁴ The Kong API gateway is stateless, hence it is possible to deploy multiple instances and distribute the requests using a load balancer.

微服务的弹性

Resilience in Microservices

分布式系统经常不言而喻的真理之一是，在大部分时间里，系统都在运行没有灾难性的错误。网络快速可靠，机器和磁盘很少崩溃，用于托管微服务、消息传递和数据库的基础平台非常强大。当系统处理低请求量并拥有充足的 CPU、内存和网络带宽来让用户感到非常满意时，尤其如此。当然，您的系统仍然必须为可能发生的间歇性故障做好准备，通常是在最不方便的时候！

One of the frequently unstated truisms of distributed systems is that, for the vast amount of the time, systems operate without catastrophic errors. Networks are fast and reliable, machines and disks rarely crash, the foundational platforms you use for hosting microservices and messaging and databases are incredibly robust. This is especially true when systems are handling low request volumes, and have plenty of CPU, memory, and network bandwidth to keep their users extremely happy. Of course, your system still has to be prepared for intermittent failures that will occur, usually at the most inconvenient of times!

当请求频率和数量增加时，事情开始变得非常有趣。线程争夺处理时间，内存变得稀缺，网络连接变得饱和，并且延迟增加。这是各个微服务开始表现不可预测的时候。然后，所有的赌注都落空了。

Things start to get really fun when request frequencies and volumes increase. Threads contend for processing time, memory becomes scarce, network connections become saturated, and latencies increase. This is when individual microservices start behaving unpredictably. Then, all bets are off.

为了确保您的系统不会随着负载的增加而突然发生故障，您需要采取一些必要的预防措施。我将在以下小节中解释您需要注意的问题的性质以及可用的解决方案。

To ensure your systems don’t fail suddenly as loads increase, there are a number of necessary precautions you need to take. I’ll explain the nature of the problems that you need to be aware of, and the solutions available, in the following subsections.

级联故障

Cascading Failures

图 9-5描述了一个简单的微服务架构。请求到达微服务A，进行处理这个请求，它会调用微服务B，微服务B又调用微服务C。一旦微服务C响应，B就可以将结果返回给A，A又可以响应客户端。图中的数字代表单个请求的序列。

Figure 9-5 depicts a simple microservices architecture. A request arrives at microservice A. To process this request, it calls microservice B, which in turn calls microservice C. Once microservice C responds, B can return the results to A, which in turn can respond to the client. The numbers in the figure represent this sequence for an individual request.

图 9-5。具有依赖关系的微服务

现在我假设微服务 A 上的请求负载增加。这意味着A将给B施加更多的负载，这将反过来会给 C 施加更多负载。由于某种原因，例如处理能力不足或数据库争用，这会导致微服务 C 的响应时间增加，从而对 B 造成背压，导致其对 A 的响应速度更慢。

Now I’ll assume that the request load on microservice A grows. This means A will exert more load on B, which will in turn exert more load on C. For some reason, such as lack of processing capacity or database contention, this causes the response times from microservice C to increase, which creates back pressure on B and causes it to respond more slowly to A.

如果增加的负载持续一段时间，微服务 A 和 B 中的线程将被阻塞，等待下游处理处理请求。让我们假设微服务 C 变得过载 - 也许请求模式导致频繁更新的密钥上的数据库死锁，或者与 C 数据库的网络连接变得不稳定。在过载状态下，响应时间会增加，并且 B 的线程会被阻塞以等待结果。还记得第 2 章中提到的，应用程序服务器具有固定大小的线程池。一旦 B 中的所有线程都占用了对 C 的调用，如果请求继续大量到达，它们将排队直到有线程可用。从 B 到 A 的响应时间开始增长，瞬间所有 A 的线程都将被阻塞，等待 B 响应。

If the increased load is sustained for a period of time, threads in the microservices A and B are blocked waiting for requests to be handled by downstream processing. Let’s assume microservice C becomes overloaded—perhaps the request pattern causes database deadlocks on frequently updated keys, or the network connection to C’s database becomes unstable. In an overloaded state, response times increase and B’s threads become blocked waiting for results. Remember from Chapter 2, application servers have fixed-size thread pools. Once all threads in B are occupied making calls to C, if requests continue to arrive at high volumes, they will be queued until a thread is available. Response times from B to A start to grow, and in an instant all of A’s threads will be blocked waiting for B to respond.

在这个阶段，事情可能会开始破裂。TCP 请求将超时并向调用者抛出错误。由于相关服务过载，新连接将被拒绝。如果内存耗尽，或者增加的负载发现了测试期间未发现的细微错误，微服务可能会失败。这些错误会通过调用链产生连锁反应或级联。在图9-5的示例中，C的响应速度较慢会导致向A和B的请求失败。

At this stage, things will likely start to break. TCP requests will time out and throw an error to the caller. New connections will be refused as the dependent service is overloaded. Microservices may fail if memory is exhausted, or the increased load uncovers subtle bugs that weren’t revealed during testing. These errors ripple, or cascade back through the call chain. In the example in Figure 9-5, the slow responses from C can cause requests to A and B to fail.

级联故障的阴险本质在于它们是由依赖服务的缓慢响应时间触发的。如果下游服务由于系统崩溃或短暂的网络故障而简单地失败或不可用，调用者会立即收到错误并可以做出相应的响应。对于逐渐减慢的服务来说，情况并非如此。请求返回结果，只是响应时间较长。如果不堪重负的组件继续受到请求的轰炸，它就没有时间恢复，响应时间会继续增长。

The insidious nature of cascading failures is that they are triggered by slow response times of dependent services. If a downstream service simply fails or is unavailable due to a system crash or transient network failure, the caller gets an error immediately and can respond accordingly. This is not the case with services that gradually slow down. Requests return results, just with longer response times. If the overwhelmed component continues to be bombarded with requests, it has no time to recover and response times continue to grow.

这种情况通常会因客户端在请求失败后立即重试操作而加剧，如以下代码片段所示：

This situation is often exacerbated by clients that, upon request failure, immediately retry the operation, as illustrated in the following code snippet:

int 重试 = RETRY_COUNT;
while (重试 > 0) {
   尝试 {
       调用DependentService();
       返回真；
    } catch (RemoteCallException ex) {
        日志错误（e）；
        重试次数=重试次数-1；
  }
  返回假；

int retries = RETRY_COUNT;
while (retries > 0) {
   try {
       callDependentService();
       return true;
    } catch (RemoteCallException ex) {
        logError(e);
        retries = retries – 1;
  }
  return false;

立即重试只是维持不堪重负的微服务上的负载，结果非常可预测，即另一个例外。过载情况不会在几毫秒内消失。事实上，它们可能会持续数秒甚至数分钟。重试只是保持压力。

Immediate retries simply maintain the load on the overwhelmed microservice, with very predictable results, namely another exception. Overload situations don’t disappear in a few milliseconds. In fact, they are likely to persist for many seconds or even minutes. Retries just keep the pressure on.

重试示例可以通过指数退避等技术进行改进，即在重试之间插入不断增长的延迟。这可能有助于缓解下游过载，但延迟成为调用者经历的延迟的一部分，这通常无济于事。

The retry example can be improved by techniques such as exponential backoff, namely inserting a growing delay between retries. This potentially can help relieve the downstream overload, but the delay becomes part of the latency experienced by the caller, which often doesn’t help matters.

级联故障在分布式系统中很常见。无论是由不堪重负的服务引起的，还是错误情况（例如错误或网络问题）引起的，您都需要采取明确的步骤来防范它们。

Cascading failures are common in distributed systems. Whether caused by overwhelmed services, or error conditions such as bugs or network problems, there are explicit steps you need to take to guard against them.

快速失败模式

Fail fast pattern

服务速度慢的核心问题是它们长时间利用系统资源来处理请求。请求线程将被停止，直到收到响应。例如，假设我们有一个通常在 50 毫秒内响应的 API。这意味着每个线程每秒可以处理大约 20 个请求。如果一个请求由于响应时间异常而停滞 3 秒，则可以处理的请求数为 (3 × 20) – 1 = 59 个。

The core problem with slow services is that they utilize system resources for requests for extended periods. A requesting thread is stalled until it receives a response. For example, let’s assume we have an API that normally responds within 50 ms. This means each thread can process around 20 requests per second. If one request is stalled for 3 seconds due to an outlier response time, then that’s (3 × 20) – 1 = 59 requests that could have been processed.

即使为微服务设计了最好的 API，也会出现异常响应。实际工作负载表现出长尾响应时间曲线，如图9-6所示。少数请求所花费的时间比平均响应时间要长得多，有时是 20 或 100 倍。这可能有多种原因。服务器中的垃圾收集、数据库争用、过多的上下文切换、系统页面错误和网络请求丢失都是造成这种长尾的常见原因。

Even with the best designed APIs for a microservice, there will be outlier responses. Real workloads exhibit a long-tail response time profile, as illustrated in Figure 9-6. A small number of requests takes significantly longer—sometimes 20 or a 100 times more—than the average response time. This can be for a number of reasons. Garbage collection in the server, database contention, excessive context switching, system page faults, and dropped network requests are all common causes for this long tail.

图 9-6。典型的长尾响应时间

正如您从该图中观察到的，绝大多数请求的响应时间都很短，这很好。然而，相当多的时间超过一秒，而一小部分则慢得多，实际上超过 4 秒。

As you can observe from this graph, the vast majority of requests have low response times, which is great. However, a significant number are over one second and a small number much, much slower—over 4 seconds, in fact.

我们可以使用百分位数来量化缓慢请求的百分比。与平均值相比，百分位数可以更丰富、更准确地了解微服务的响应时间。例如，如果我们测量响应时间并计算预期负载下的百分位数，我们可能会得到以下结果：

We can quantify the percentage of slow requests using percentiles. Percentiles give a far richer and more accurate view of response times from a microservice than averages. For example, if we measure response times and calculate percentiles under expected loads, we may get the following:

• P50：200毫秒

• P50: 200 milliseconds

• P95：1,200 毫秒

• P95: 1,200 milliseconds

• P99：3,000 毫秒

• P99: 3,000 milliseconds

这意味着 50% 的请求在 200 毫秒内得到满足，95% 在 1,200 毫秒内得到满足，99% 在 3,000 毫秒内得到满足。这些数字总的来说看起来相当不错。但假设我们的 API 每天处理 2 亿个请求（每秒大约 2,314 个请求）。这意味着 1%（即 200 万个请求）花费的时间超过 3 秒，比第 50 个百分位（中位数）慢 15 倍。考虑到我们在图 9-6中看到的长尾响应时间模式，一些请求将明显长于 3 秒。

This means that 50% of requests are served in less than 200 milliseconds, 95% are served within 1,200 milliseconds, and 99% percent within 3,000 milliseconds. These numbers in general look pretty good. But let’s assume our API handles 200 million requests per day (approximately 2,314 requests per second). This means 1%, or 2 million requests, take greater than 3 seconds, which is 15 times slower than the 50th percentile (the median). And some requests will be significantly longer than 3 seconds given the long-tail response time pattern we see in Figure 9-6.

无论从技术上还是从客户参与角度来说，较长的响应时间从来都不是一件好事。事实上，许多研究表明较长的响应时间会对系统使用产生负面影响。例如，BBC 报道称，页面加载时间每增加一秒，用户数量就会减少 10%。快速、稳定的响应时间对业务来说非常有利，实现这一目标的方法之一是减少长尾。这还会降低服务的总体平均响应时间，因为少数缓慢响应会严重影响平均值。

Long response times are never good things, technically or for client engagement. In fact, many studies have shown how longer response times have negative effects on system usage. For example, the BBC reported that it sees 10% less users for every additional second a page takes to load. Fast, stable response times are great for business, and one way to achieve this is to reduce the long tail. This also has the effect of decreasing the overall average response time for a service, as the average is skewed heavily by a small number of slow responses.

消除较长响应时间的常见方法是快速失败。有两种主要方法可以实现这一目标：

A common way to eliminate long response times is to fail fast. There are two main ways to achieve this:

• 当请求花费的时间超过某个预定义的时间限制时，客户端不会等待其完成，而是向其调用者返回错误。这会释放与请求关联的线程和其他资源。

• When a request takes longer than some predefined time limit, instead of waiting for it to complete, the client returns an error to its caller. This releases the thread and other resources associated with the request.

• 在服务器上启用限制。如果请求负载超过某个阈值，则立即使请求失败并显示 HTTP 503 错误。这向客户端表明该服务不可用。

• Enable throttling on a server. If the request load exceeds some threshold, immediately fail the request with an HTTP 503 error. This indicates to the client that the service is unavailable.

这些策略的具体实施方式取决于技术的具体情况。例如，发出 HTTP 请求的客户端可以配置TCP 读取超时。这指定客户端应等待接收服务器响应的时间。在图 9-6的示例中，我们可以将读取超时配置为 P99 值，即 3 秒或稍高一些。然后，如果客户端在读取超时时间内没有收到任何响应，则会引发异常。在 Java 中，它是一个java.net.SocketTimeoutException.

Exactly how these strategies are put into action is extremely technology-specific. For example, a client making an HTTP request can configure the TCP read timeout. This specifies how long a client should wait for to receive a response from the server. In our example in Figure 9-6, we could configure the read timeout to the P99 value, namely 3 seconds or a little higher. Then, if a client hasn’t received any response within the read timeout period, an exception is raised. In Java, it’s a java.net.SocketTimeoutException.

节流或速率限制是许多负载均衡器和 API 网关技术中提供的功能。什么时候达到某些定义的限制时，负载均衡器将简单地拒绝请求，从而保护其控制的资源免受过载。这使得服务能够以一致的低响应时间处理请求。还可以在微服务内部实现一些轻量级监控逻辑来实现限制。您可以保留正在进行的请求的计数，如果计数超过定义的最大值，则新请求将被拒绝。稍微复杂一点的方法可以使用滑动窗口算法来跟踪平均响应时间或 P99 等指标。如果感兴趣的指标正在增加或超过某个定义的阈值，则请求可以立即被拒绝。

Throttling, or rate limiting, is a feature available in many load balancers and API gateway technologies. When some defined limits are reached, the load balancer will simply reject requests, protecting the resources it controls from overload. This enables the service to process requests with consistent low response times. It’s also possible to implement some lightweight monitoring logic inside your microservice to implement throttling. You might keep a count of in-flight requests, and if the count exceeds a defined maximum, new requests are rejected. A slightly more sophisticated approach could track a metric like the average response time, or P99s, using a sliding window algorithm. If the metric of interest is increasing, or exceeds some defined threshold, again requests can be immediately rejected.

请求失败时还需要考虑一件事。微服务的一个原则是故障隔离。这意味着部分系统的故障不会导致整个应用程序不可用。可以继续处理请求，但功能会有所下降。

There’s one more thing to consider when failing requests. A principle of microservices is fault isolation. This means the failure of part of the system doesn’t make the whole application unavailable. Requests can continue to be processed, but with some degraded capabilities.

需要考虑的一个关键问题是是否有必要将错误传播回原始调用者。或者是否可以发送一些预设的默认响应来掩盖请求未正确处理的事实？例如，当您登录流媒体视频服务时，第一页将显示您的观看列表，以便您可以尽快返回到您喜爱的节目。但是，如果检索观看列表的请求失败或花费太长时间，则可以返回热门“您可能喜欢的节目”的默认集合。该应用程序仍然可用。

A key thing to consider is whether it is necessary to propagate the error back to the original caller. Or can some canned, default response be sent that masks the fact that the request was not correctly processed? For example, when you sign into a streaming video service, the first page will show your watchlist so you can return to your favorite shows as quickly as possible. If, however, the request to retrieve your watchlist fails, or takes too long, a default collection of popular “shows you might like” can be returned. The application is still available.

这种方法对于暂时性、短暂的故障非常有效。当用户再次发出请求时，问题可能会得到解决。用户很可能都没有注意到。然而，一些暂时性错误不会在一两秒内解决。这就是您需要更强大的方法的时候。

This approach works really well for transient, ephemeral failures. By the time the request is issued again by the users, the problem will probably be resolved. And there’s a good chance the user won’t even have noticed. Some transient errors, however, don’t resolve in a second or two. That’s when you need a more robust approach.

断路器模式

Circuit breaker pattern

如果微服务开始抛出错误由于过载情况或不稳定的网络，继续尝试向 API 发送请求没有任何意义。与其快速失败（仍然会导致超时延迟），不如立即停止发送进一步的请求，并留出一些时间来解决错误情况。这可以使用断路器模式来实现，该模式可以保护远程端点在发生某些错误情况时不会被淹没。

If a microservice starts to throw errors due to an overload situation, or a flaky network, it makes little sense to keep trying to send requests to the API. Rather than failing fast, which still incurs a timeout delay, it is better to back off immediately from sending further requests and allow some time for the error situation to resolve. This can be achieved using the circuit breaker pattern, which protects remote endpoints from being overwhelmed when some error conditions occur.

就像在电气系统中一样，客户端可以使用断路器来保护服务器免受过载。断路器配置为监视某些条件，例如来自端点的错误响应率或每秒发送的请求数。如果达到配置的阈值（例如，25% 的请求抛出错误），则会触发断路器。这会使断路器进入一种OPEN状态，在该状态下，所有调用都会立即返回错误，并且不会尝试调用不稳定或不可用的端点。

Just like in electrical systems, clients can use a circuit breaker to protect a server from overload. The circuit breaker is configured to monitor some condition, such as error response rates from an endpoint, or the number of requests sent per second. If the configured threshold is reached—for example, 25% of requests are throwing errors—the circuit breaker is triggered. This moves the circuit breaker into an OPEN state, in which all calls return with an error immediately, and no attempt is made to call the unstable or unavailable endpoint.

然后，断路器会拒绝所有调用，直到某个适当配置的超时期限到期。在此阶段，断路器进入状态HALF_OPEN。现在，断路器允许向受保护端点发出客户端调用。如果请求仍然失败，则重置超时期限并且断路器保持打开状态。但是，如果请求成功，断路器将转换到状态CLOSED，并且请求开始流向目标端点。该方案如图9-7所示。

The circuit breaker then rejects all calls until some suitably configured timeout period expires. At that stage, the circuit breaker moves to the HALF_OPEN state. Now, the circuit breaker allows client calls to be issued to the protected endpoint. If the requests still fail, the timeout period is reset and the circuit breaker stays open. However, if the request succeeds, the circuit breaker transitions to the CLOSED state and requests start to flow to the target endpoint. This scheme is illustrated in Figure 9-7.

图 9-7。断路器模式

断路器对于减少几乎肯定会失败的操作所使用的资源至关重要。客户端很快就会失败，OPEN断路器通过确保请求不会到达服务器来减轻不堪重负的服务器的负载。对于过载的服务，这创造了稳定的机会。当服务（希望）恢复时，断路器会自动重置并恢复正常运行。

Circuit breakers are essential to reduce the resources utilized for operations that are almost certain to fail. The client fails fast, and the OPEN circuit breaker relieves load on an overwhelmed server by ensuring requests do not reach it. For overloaded services, this creates an opportunity to stabilize. When the service (hopefully) recovers, the circuit breaker resets automatically and normal operations resume.

有许多库可用于将断路器合并到您的应用程序中。以下代码示例说明了一种流行的 Python 库CircuitBreaker 。您只需使用 修饰要保护的外部调用@circuit，并指定要设置的参数值以自定义断路器行为。在本例中，我们在检测到连续 20 次故障后触发断路器，断路器保持打开状态 5 秒，直到转换为半打开状态：

There are numerous libraries available for incorporating circuit breakers into your applications. One popular library for Python, CircuitBreaker, is illustrated in the following code example. You simply decorate the external call you want to protect with @circuit, and specify the value of the parameters you wish to set to customize the circuit breaker behavior. In this example, we trigger the circuit breaker after 20 successive failures are detected, and the circuit breaker stays open for 5 seconds until it transitions to the half open state:

从断路器输入电路

@电路（failure_threshold = 20，expected_exception = RequestException，
         恢复超时=5)
def api_call():

from circuitbreaker import circuit

@circuit(failure_threshold=20,expected_exception=RequestException,
         recovery_timeout=5)
def api_call():

断路器对于故障隔离非常有效。它们保护客户端免受相关服务的错误操作的影响，并允许服务恢复。在读取繁重的场景中，当断路器打开时，请求通常可以返回默认或缓存的结果。这有效地向客户端隐藏了故障，并且不会降低服务吞吐量和响应时间。确保将断路器触发器绑定到监控中和记录基础设施，以便诊断故障原因。

Circuit breakers are highly effective for fault isolation. They protect clients from faulty operations of dependent services and allow services to recover. In read-heavy scenarios, requests can often return default or cached results when the circuit breaker is open. This effectively hides the fault from clients and doesn’t degrade service throughput and response times. Ensure you tie circuit breaker triggers into your monitoring and logging infrastructure so that the cause of faults can be diagnosed.

// 配置舱壁
BulkheadConfig 配置 = BulkheadConfig.custom()
            .maxConcurrentCalls(150)
            .maxWaitDuration(持续时间.ofSeconds(1))
            。建造（）;
BulkheadRegistry 注册表 = BulkheadRegistry.of(config);
// 创建舱壁
隔板 newOrderBulkhead =registry.bulkhead("newOrder");

// configure the bulkhead
BulkheadConfig config = BulkheadConfig.custom()
            .maxConcurrentCalls(150)
            .maxWaitDuration(Duration.ofSeconds(1))
            .build();
BulkheadRegistry registry = BulkheadRegistry.of(config);
// create the bulkhead
Bulkhead newOrderBulkhead = registry.bulkhead("newOrder");

// 用舱壁装饰 OrderService.newOrder 方法
供应商<OrderOutcome> orderSupplier = () ->
    OrderService.newOrder(OrderInfo);
// 使用舱壁配置装饰 NewOrder
供应商<OrderOutcome> bukheadOrderSupplier =
    舱壁.decorateSupplier（舱壁，orderSupplier）；

// decorate the OrderService.newOrder method with the bulkhead 
Supplier<OrderOutcome> orderSupplier = () -> 
    OrderService.newOrder(OrderInfo);
// decorate NewOrder with the bulkhead configuration
Supplier<OrderOutcome> bukheadOrderSupplier = 
    bulkhead.decorateSupplier(bulkhead, orderSupplier);

服务器：
  雄猫：
    线程：
      最大：200
弹性4j.舱壁：
  实例：
    订单服务：
      最大并发调用数：150
      最大等待持续时间：1000ms

server:
  tomcat:
    threads:
      max: 200
resilience4j.bulkhead:
  instances:
    OrderService:
      maxConcurrentCalls: 150
      maxWaitDuration: 1000ms

@Bulkhead（名称=“OrderService”，fallbackMethod=“newOrderBusy”）
   public OrderOutcome newOrder(OrderInfo inf){// 详情省略}

@Bulkhead(name = "OrderService", fallbackMethod = “newOrderBusy”)
   public OrderOutcome newOrder(OrderInfo inf){// details omitted}

总结和延伸阅读

Summary and Further Reading

拥抱微服务需要您采用新的设计和开发实践来创建一系列细粒度、有凝聚力的组件，以满足您的应用程序需求。此外，您还需要面对分布式系统的新机遇和复杂性。如果您采用微服务，您就别无选择。

Embracing microservices requires you to adopt new design and development practices to create a collection of fine-grained, cohesive components to satisfy your application requirements. In addition, you also need to confront the new opportunities and complexities of distributed systems. If you adopt microservices, you simply have no choice.

本章简要概述了微服务的动机及其所能带来的优势。在本书中，独立扩展单个微服务以满足不断增长的需求的能力通常是无价的。

This chapter has given a brief overview of the motivations for microservices and the advantages they can afford. In the context of this book, the ability to independently scale individual microservices to match increasing demand is often invaluable.

微服务经常耦合，需要通信才能满足单个请求。这使得它们容易遭受级联故障。当微服务开始返回响应时间不断增加的请求时，就会发生这种情况——例如，由于请求过载或暂时性网络错误而导致。缓慢的响应时间会导致调用服务出现背压，最终其中一个服务出现故障可能会导致所有依赖服务崩溃。

Microservices are frequently coupled, needing to communicate to satisfy a single request. This makes them susceptible to cascading failures. These occur when a microservice starts to return requests with increasing response times—caused, for example, by an overload in requests or transient network errors. Slow response times cause back pressure in the calling services, and eventually a failure in one can cause all dependent services to crash.

避免级联故障的模式包括使用超时和断路器快速失败。这些本质上为受压的微服务提供了恢复和阻止级联故障发生的时间。舱壁图案的意图相似。它可用于确保对微服务中的一个 API 的请求在请求突发期间不会利用所有可用资源。通过对应用服务器中某个特定API可以请求的线程数设置最大限制，可以保证其他API的处理能力。

Patterns for avoiding cascading failures include failing fast using timeouts and circuit breakers. These essentially give the stressed microservice time to recover and stop cascading failures from occurring. The bulkhead pattern is similar in intent. It can be used to ensure requests to one API in a microservice don’t utilize all available resources during a request burst. By setting a maximum limit on the number of threads in the application server a particular API can demand, processing capacity for other APIs can be guaranteed.

微服务是软件架构中的一个主要主题。要完整、全面地涵盖该主题，没有比 Sam Newman 的《构建微服务》第二版（O'Reilly，2021 年）更好的来源了。这将带您深入了解基于微服务的系统的设计、开发和部署。

Microservices are a major topic in software architecture. For a complete and comprehensive coverage of the topic, there is no better source than Sam Newman’s Building Microservices, 2nd Edition (O’Reilly, 2021). This will take you on an in-depth journey following the design, development, and deployment of microservices-based systems.

微服务需要开发过程的广泛自动化。Jez Humble 和 David Farley（Addison-Wesley Professional）撰写的2011 年经典《持续交付：通过构建、测试和部署自动化实现可靠的软件发布》是全面介绍该主题的理想起点。另一个优秀的信息来源是Len Bass、Ingo Weber 和 Liming Zhu 撰写的《DevOps：软件架构师的视角》 （Addison-Wesley Professional，2015 年）。DevOps 的世界是一个快速发展且技术丰富的领域，您最喜欢的搜索引擎是查找有关构成现代 DevOps 管道的各种构建、配置、测试、部署和监控平台的信息的最佳场所。

Microservices require extensive automation of the development process. The 2011 classic Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley (Addison-Wesley Professional) is an ideal place to start for a comprehensive introduction to the topic. Another excellent source of information is DevOps: A Software Architect’s Perspective by Len Bass, Ingo Weber, and Liming Zhu (Addison-Wesley Professional, 2015). The world of DevOps is a fast-moving and technologically rich domain, and your favorite search engine is the best place to find information on the various build, configuration, test, deployment, and monitoring platforms that comprise modern DevOps pipelines.

接下来，本书的第三部分重点讨论存储层的主题。我将描述决定我们如何分布数据存储以实现可扩展性、可用性和一致性的核心原则和算法。该理论将通过研究许多广泛使用的数据库如何在分布式系统中运行以及它们采取的各种方法和架构权衡来补充。

Next, Part III of this book focuses on the topic of the storage layer. I’ll be describing the core principles and algorithms that determine how we can distribute data stores to achieve scalability, availability, and consistency. The theory will be complemented by examining how a number of widely used databases operate in distributed systems, and the various approaches and architectural trade-offs they take.

分布式数据库

Distributed Databases

我们今天构建的数据系统使那些相形见绌20 年前，关系型数据库统治地球。数据集大小和复杂性的增长是由互联网规模的应用程序推动的。它们为数千万用户创建和管理大量异构数据。例如，这包括用户个人资料、用户偏好、行为数据、图像和视频、销售数据、广告、传感器读数、监控数据等等。许多数据集太大而无法容纳在一台机器上。

The data systems we build today dwarf those of 20 years ago, when relational databases ruled the earth. This growth in data set size and complexity has been driven by internet-scale applications. These create and manage vast quantities of heterogeneous data for literally tens of millions of users. This includes, for example, user profiles, user preferences, behavioral data, images and videos, sales data, advertising, sensor readings, monitoring data, and much more. Many data sets are simply far too big to fit on a single machine.

这就需要进化数据库引擎来管理大量分布式数据。新一代关系型和非关系型数据库平台已经出现，具有广泛的竞争能力，旨在满足不同的用例和可扩展性要求。同时，低成本、功能强大的硬件的发展使得在数百甚至数千个节点和磁盘上经济高效地分发数据成为可能。这增强了可扩展性，并通过复制数据提高了可用性。

This has necessitated the evolution of database engines to manage massive collections of distributed data. New generations of relational and nonrelational database platforms have emerged, with a wide range of competing capabilities aimed at satisfying different use cases and scalability requirements. Simultaneously, the development of low-cost, powerful hardware has made it possible to cost-effectively distribute data across literally hundreds or even thousands of nodes and disks. This enhances both scalability and, by replicating data, availability.

数据库引擎创新的另一个主要驱动力是当今互联网上的应用程序需求不断变化的性质。关系数据库的固有优势，即事务和一致性，是以性能为代价的，而这在 Twitter 和 Facebook 等网站上并不总是合理的。这些并不要求每个用户始终看到相同的版本，例如我的推文或时间线更新。谁在乎我的美味晚餐的最新照片是否立即被我的一些粉丝和朋友看到，而其他人则必须等待几秒钟才能欣赏我正在吃的巧妙菜肴？

Another major driver of database engine innovation has been the changing nature of the application requirements that populate the internet today. The inherent strengths of relational databases, namely transactions and consistency, come at a performance cost that is not always justified in sites like Twitter and Facebook. These don’t have requirements for every user to always see the same version of, for example, my tweets or timeline updates. Who cares if the latest photo of my delicious dinner is seen immediately by some of my followers and friends, while others have to wait a few seconds to admire the artful dish I’m consuming?

对于数万到数百万的用户，可以放松关系数据库支持的各种数据限制并获得增强的性能和可扩展性。这使得能够创建新的非关系数据模型和本机分布式数据库引擎，旨在支持当今应用程序的各种用例。当然，这也是需要权衡的。这些体现在数据库支持的功能范围及其编程模型的复杂性上。

With tens of thousands to millions of users, it is possible to relax the various data constraints that relational databases support and attain enhanced performance and scalability. This enables the creation of new, nonrelational data models and natively distributed database engines, designed to support the variety of use cases for today’s applications. There are trade-offs, of course. These manifest themselves in the range of features a database supports and the complexity of its programming model.

扩展关系数据库

Scaling Relational Databases

支持关系模型和 SQL 查询语言的数据库代表了一些最成熟、稳定和强大的数据库当今存在的软件平台。您会发现关系数据库潜伏在您能想象到的各种类型的应用程序领域的系统后面。它们是极其复杂且极其成功的技术。

Databases that support the relational model and SQL query language represent some of the most mature, stable, and powerful software platforms that exist today. You’ll find relational databases lurking behind systems in every type of application domain you can imagine. They are incredibly complex and amazingly successful technologies.

关系数据库技术是在数据集按照今天的标准相对较小的时候设计和成熟的，并且数据库可以在单台机器上运行。随着数据集的增长，扩展数据库的方法也随之出现。我将在以下小节中通过一些示例简要介绍这些内容。

Relational database technology was designed and matured when data sets were relatively small by today’s standards, and the database could run on a single machine. As data sets have grown, approaches to scale databases have emerged. I’ll briefly cover these with some examples in the following subsections.

扩大

Scaling Up

关系型数据库的设计在单台机器上运行，这使得可以利用共享内存和磁盘来存储数据和处理查询。这使得数据库引擎可以定制为在具有多个 CPU、磁盘和大型共享内存的计算机上运行。数据库引擎可以利用这些资源并行执行数千个查询，以提供极高的吞吐量。

Relational databases were designed to run on a single machine, which enables shared memory and disks to be exploited to store data and process queries. This makes it possible for database engines to be customized to run on machines with multiple CPUs, disks, and large shared memories. Database engines can exploit these resources to execute many thousands of queries in parallel to provide extremely high throughput.

图 10-1描述了扩展场景。数据库迁移到新的、更强大的（虚拟）硬件。虽然数据库管理魔法可以执行迁移并调整数据库配置以有效利用新资源，但应用程序代码不需要任何更改。

Figure 10-1 depicts the scale-up scenario. The database is migrated to new, more powerful (virtual) hardware. While there is database administration magic to perform the migration and tune the database configuration to effectively exploit the new resources, the application code should require no changes.

这种方法有三个主要缺点：

There are three main downsides to this approach:

成本

Cost

随着提供的计算资源的增长，硬件成本往往呈指数级增长。

Hardware costs tend to grow exponentially as the computational resources offered grow.

可用性

Availability

尽管功能强大，但您仍然拥有一个数据库节点。如果它变得不可用，则您的系统已关闭。存在多种高可用性 (HA) 解决方案，它们提供了检测数据库备份副本不可用性和故障转移的机制。许多 HA 解决方案都依赖于数据库供应商。

You still have a single database node, albeit a powerful one. If it becomes unavailable, your system is down. A multitude of high availability (HA) solutions exist that offer mechanisms to detect unavailability and failover to a backup copy of the database. Many HA solutions are database vendor dependent.

生长

Growth

如果您的数据库继续增长，则不可避免地会再次迁移到更强大的硬件。

If your database continues to grow, another migration to more powerful hardware is inevitable.

图 10-1。关系数据库扩展场景示例

在许多应用中，扩大规模确实很有吸引力。尽管如此，在大批量应用中，有两种常见情况会导致扩展出现问题。首先，数据库的增长超出了单个节点的处理能力。其次，需要低延迟的数据库访问来为遍布全球的客户提供服务。穿越洲际网络根本无法解决这个问题。

Scaling up is indeed attractive in many applications. Still, in high-volume applications, there are two common scenarios in which scaling up becomes problematic. First, the database grows to exceed the processing capability of a single node. Second, low latency database accesses are required to service clients spread around the globe. Traversing intercontinental networks just doesn’t cut it.

在这两种情况下，都需要分发数据库。

In both cases, distributing a database is necessary.

横向扩展：只读副本

Scaling Out: Read Replicas

增加数据库容量的常见第一步处理能力是使用只读副本进行横向扩展。您将一个或多个节点配置为主数据库的只读副本。主数据库节点称为主节点，只读副本称为辅助节点。辅助数据库维护主数据库的副本。写入只能写入主数据库，然后所有更改都会异步复制到辅助数据库。辅助节点可能位于不同的数据中心或不同的大陆，以支持全球客户。

A common first step to increasing a database’s processing capacity is to scale out using read replicas. You configure one or more nodes as read replicas of the main database. The main database node is known as the primary, and read replicas are known as secondaries. The secondaries maintain a copy of the main database. Writes are only possible to the primary, and all changes are then asynchronously replicated to secondaries. Secondaries may be physically located in different data centers or different continents to support global clients.

该架构如图 10-2所示。

This architecture is shown in Figure 10-2.

图 10-2。通过读复制进行分发

此方法通过将所有读取定向到只读副本来增强可扩展性。¹因此，它对于必须支持读取繁重工作负载的应用程序非常有效。可以通过添加更多辅助节点来扩展读取，从而减少主节点上的负载。这使其能够更有效地处理写入。此外，如果主数据库由于暂时性故障而变得不可用，则定向到辅助数据库的读取请求不会中断。

This approach enhances scalability by directing all reads to the read replicas.¹ It is hence highly effective for applications that must support read-heavy workloads. Reads can be scaled by adding more secondaries, reducing the load on the primary. This enables it to more efficiently handle writes. In addition, if the primary becomes unavailable due to a transient failure, read requests directed to secondaries are not interrupted.

由于数据写入主数据库和成功复制到辅助数据库之间存在延迟，因此客户端有可能从辅助数据库读取过时的数据。因此应用程序必须意识到这种可能性。在正常操作中，更新主数据库和辅助数据库之间的时间应该很短，例如几毫秒。这个时间窗口越小，出现陈旧读取的可能性就越小。

As there is a delay between when data is written to the primary and then successfully replicated to the secondaries, there is a chance that clients may read stale data from secondaries. Application must therefore be aware of this possibility. In normal operations, the time between updating the primary and the secondaries should be small, for example, a few milliseconds. The smaller this time window, then the less chance there is of a stale read.

读复制和基于主/辅助的数据库架构是我将在本节和第二节中更详细地讨论的主题。以下章节。

Read replication and primary/secondary–based database architectures are topics I’ll return to in much more detail in this and the following chapters.

横向扩展：数据分区

Scale Out: Partitioning Data

将数据拆分或分区关系数据库，是一种将数据库分布在多个独立磁盘分区和数据库引擎上的技术。分区的具体支持方式是高度特定于产品的。一般来说，有两种策略：水平分区和垂直分区。

Splitting up, or partitioning data in a relational database, is a technique for distributing the database over multiple independent disk partitions and database engines. Precisely how partitioning is supported is highly product-specific. In general, there are two strategies: horizontal partitioning and vertical partitioning.

水平分区将一个逻辑表拆分为多个物理分区。根据某种分区策略将各个行分配到分区。常见的分区策略是根据行中的某些值将行分配给分区，或者在主键上使用哈希函数。如图10-3所示，您可以根据每行中region字段的值将行分配给分区。

Horizontal partitioning splits a logical table into multiple physical partitions. Individual rows are allocated to a partition based on some partitioning strategy. Common partitioning strategies are to allocate rows to partitions based on some value in the row, or to use a hash function on the primary key. As shown in Figure 10-3, you can allocate a row to a partition based on the value of the region field in each row.

图 10-3。数据库水平分区

垂直分区也称为行拆分，按行中的列对表进行分区。与标准化一样，垂直分区将一行分成一个或多个部分，但出于物理优化而非概念优化的原因。常见的策略是在静态、只读数据和动态数据之间划分行。图 10-4显示了采用此方案的库存系统的简单垂直分区。

Vertical partitioning, also known as row splitting, partitions a table by the columns in a row. Like normalization, vertical partitioning splits a row into one or more parts, but for the reasons of physical rather than conceptual optimization. A common strategy is to partition a row between static, read-only data and dynamic data. Figure 10-4 shows a simple vertical partitioning for an inventory system that employs this scheme.

图 10-4。垂直数据库分区

关系数据库引擎将为数据分区提供不同级别的支持。有些有助于在磁盘上对表进行分区。其他支持跨节点分区数据以在分布式系统中水平扩展。

Relational database engines will have various levels of support for data partitioning. Some facilitate partitioning tables on disk. Others support partitioning data across nodes to scale horizontally in a distributed system.

无论如何，关系模式的本质（数据分布在多个表中）使得设计通用的分发分区策略变得困难。数据的水平分区理想地将表分布在多个节点上。然而，如果单个请求需要访问多个节点的数据，或者连接分布式分区表的数据，则需要高水平的网络流量和请求协调。这可能不会带来您期望的性能优势。以下侧边栏简要介绍了这些问题。

Regardless, the very nature of relational schemas, with data split across multiple tables, makes it problematic to devise a general partitioning strategy for distribution. Horizontal partitions of data ideally distribute tables across multiple nodes. However, if a single request needs to access data from multiple nodes, or join data from distributed partitioned tables, a high level of network traffic and request coordination is required. This may not give the performance benefits you expect. These issues are briefly covered in the following sidebar.

示例：Oracle RAC

Example: Oracle RAC

尽管分区存在固有的问题由于关系模型和大规模 SQL 查询的复杂性，供应商在过去二十年中一直致力于横向扩展关系数据库。一个著名的例子是 Oracle 的真正应用集群 (RAC) 数据库。

Despite the inherent problems of partitioning relational models and the complexities of SQL queries at scale, vendors have worked in the last two decades to scale out relational databases. One notable example is Oracle’s Real Applications Cluster (RAC) database.

Oracle的RAC数据库于2001年发布，为大容量、高可用的系统提供分布式版本的Oracle数据库引擎。从本质上讲，Oracle 可以部署多达 100 个 Oracle 数据库引擎的集群，这些引擎都访问同一物理数据库。

Oracle’s RAC database was released in 2001 to provide a distributed version of the Oracle database engine for high-volume, highly available systems. Essentially, Oracle makes it possible to deploy a cluster of up to 100 Oracle database engines that all access the same physical database.

为了避免数据分区问题，Oracle RAC 是共享一切数据库的一个示例。集群数据库引擎访问构成 Oracle 数据库的数据文件、日志和配置文件的单个共享数据存储。对于数据库客户端来说，集群部署是透明的，并且显示为单个数据库引擎。

To avoid the data partitioning problem, Oracle RAC is an example of a shared-everything database. The clustered database engines access a single, shared data store of the data files, logs, and configuration files that comprise an Oracle database. To the database client, the clustered deployment is transparent and appears as a single database engine.

物理存储需求使用称为存储区域网络 (SAN) 的网络可访问存储解决方案可供所有节点访问。SAN 提供对 Oracle 数据库的高速网络访问。SAN 还必须提供硬件级磁盘镜像，以创建应用程序和系统数据的多个副本，以应对磁盘故障。在高负载下，SAN 可能成为瓶颈。高端 SAN 是极其专业的存储设备，购买起来非常昂贵。

The physical storage needs to be accessible to all nodes using a network-accessible storage solution known as Storage Area Network (SAN). SANs provide high-speed network access to the Oracle database. SANs also must provide hardware-level disk mirroring to create multiple copies of application and system data in order to survive disk failure. Under high load, the SAN can potentially become a bottleneck. High-end SANs are extremely specialized storage devices that are expensive beasts to acquire.

Oracle RAC 部署需要两个专有软件组件，即：

Two proprietary software components are required for Oracle RAC deployments, namely:

集群件

Clusterware

支持通信和集群数据库引擎之间的协调。例如，它管理集群节点成员资格、节点故障转移和高可用性。

Supports communications and coordination between the clustered database engines. It manages, for example, cluster node membership, node failover, and high availability.

缓存融合

Cache Fusion

启用每个中的单独缓存集群数据库节点能够有效共享，从而最大限度地减少对持久存储的访问。

Enables the individual caches in each clustered database node to be effectively shared so that accesses to the persistent store are minimized.

RAC系统的概述如图10-5所示。

An overview of a RAC system is shown in Figure 10-5.

图 10-5。Oracle RAC 概述

Oracle RAC 展示了一种扩展关系数据库的架构方法，即共享一切。它为 Oracle 部署增加了处理能力和高可用性，同时（理论上）不需要更改应用程序代码。该数据库需要多个专有的 Oracle 软件组件以及昂贵的冗余存储和互连硬件。加上 Oracle 许可成本，无论如何您都没有低成本的解决方案。

Oracle RAC illustrates one architectural approach, namely shared everything, to scaling a relational database. It adds processing capacity and high availability to an Oracle deployment while requiring (in theory anyway) no application code changes. The database requires multiple proprietary Oracle software components and expensive redundant storage and interconnect hardware. Add Oracle license costs, and you don’t have a low-cost solution by any means.

过去 20 年来，许多 Oracle 客户都采用了这项技术。它已经成熟且经过验证，但从当今的技术格局来看，它所基于的架构以高成本提供有限的按需可扩展性。另一种方案，即利用广泛可用的低成本商品计算节点和存储的无共享架构，是我今后重点关注的方法。

Many Oracle customers have adopted this technology in the last 20 years. It’s mature and proven, but through the lens of today’s technology landscape, is based on an architecture that offers limited on-demand scalability at high costs. The alternative, namely a shared-nothing architecture that exploits the widely available low-cost commodity compute nodes and storage is the approach I’ll focus on going forward.

向 NoSQL 的转变

The Movement to NoSQL

我没有足够的勇气去尝试和构建一个连贯的叙述，描述了创造新一代 NoSQL 数据库技术的力量。²我个人的倾向是，这种创新是由 2000 年代初开始积聚势头的多种原因共同推动的。排名不分先后，其中一些原因是：

I’m not brave enough to try and construct a coherent narrative describing the forces that brought about the creation of a new generation of NoSQL database technologies.² My personal inclination is that this innovation was driven by a confluence of reasons that started to gather momentum in the early 2000s. In no particular order, some of these reasons were:

• 开发功能强大、低成本的商品硬件，包括多核 CPU、更快、更大的磁盘以及更高的网络速度。

• The development of powerful, low-cost, commodity hardware, including multicore CPUs, faster, larger disks, and increased network speeds.

• 处理非结构化数据类型以及快速发展的业务和数据模型的应用程序的出现。关系追随者的“一刀切”方法不再适用于这些新用例。

• The emergence of applications that dealt with unstructured data types and rapidly evolving business and data models. No longer was the “one size fits all” approach of relational adherents applicable to these new use cases.

• 对面向互联网的应用程序的可扩展性和可用性的需求不断增加。

• Increased need for scalability and availability for internet-facing applications.

• 收集原始数据并将其用于新的业务洞察和分析的新机会。

• New opportunities to gather raw data and utilize this for new business insights and analytics.

结合我在本章中描述的针对海量数据集扩展关系数据库的复杂性，现在正是需要新的数据库范例的时候。这种创新所需的许多数据库和分布式系统理论都是已知的，这为一系列新数据库平台的出现创造了肥沃的土壤。

Combined with the complexities of scaling relational databases for massive data sets that I’ve described in this chapter, the time was rife for a new database paradigm. Much of the database and distributed systems theory that was needed for such innovation was known, and this created fertile ground for the emergence of a whole collection of new database platforms.

NoSQL 数据库生态系统是为了应对 2000 年代初不断变化的业务和技术环境而蓬勃发展的，它绝不是一个同质的地方。出现了几种不同的方法并在一定程度上在各种（主要是开源）数据库中实现。但总的来说，NoSQL 运动的核心特征是：

The NoSQL database ecosystem that blossomed to address the evolving business and technological landscape of the early 2000s is by no means a homogeneous place. Several different approaches emerged and were implemented to some extent in various (mostly open source) databases. In general, however, the core characteristics of the NoSQL movement are:

• 可以轻松演化的简化数据模型

• Simplified data models that can be easily evolved

• 对联接的支持有限或不支持的专有查询语言

• Proprietary query languages with limited or no support for joins

• 对低成本商用硬件上的水平扩展的本机支持

• Native support for horizontal scaling on low-cost, commodity hardware

我将在下面的小节中依次介绍这些特征。但在此之前，请考虑一下：如果没有执行类似 JOIN 查询的能力，NoSQL 数据库如何生存？答案在于如何使用 NoSQL 建模数据。

I’ll look at each of these characteristics in turn in the following subsections. But before that, consider this: how do NoSQL databases survive without the capability to execute JOIN-like queries? The answer lies in how you model data with NoSQL.

数据模型标准化，如受关系数据库的鼓励，提供了一种经过验证的问题域建模技术。它为每个数据项创建具有单个条目的模型，可以在需要时引用。更新只需要修改规范数据引用，然后更新即可用于引用该数据的所有查询。由于 SQL 和联接的强大功能，您不必过多考虑立即和将来访问数据的所有奇怪而美妙的方式。您的规范化模型（理论上）应该支持应用程序域的任何合理查询，而 SQL 使之成为可能。

Data model normalization, as encouraged by relational databases, provides a proven technique for modeling the problem domain. It creates models with a single entry for every data item, which can be referenced when needed. Updates just need to modify the canonical data reference, and the update is then available to all queries that reference the data. Due to the power of SQL and joins, you don’t have to think too hard about all the weird and wonderful ways the data will be accessed, both immediately and in the future. Your normalized model should (in theory) support any reasonable query for the application domain, and SQL is there to make it possible.

使用 NoSQL，重点从问题域建模转变为解决方案域建模。解决方案领域建模要求您考虑应用程序必须支持的常见数据访问模式，并设计支持这些访问的数据模型。对于读取数据，这意味着您的数据模型必须预先加入服务请求所需的数据。本质上，您生成了关系建模者认为的非规范化数据模型。您正在以灵活性换取效率。

With NoSQL, the emphasis changes from problem domain modeling to modeling the solution domain. Solution domain modeling requires you to think about the common data access patterns the application must support, and to devise a data model that supports these accesses. For reading data, this means your data model must prejoin the data you need to service a request. Essentially, you produce what relational modelers deem a denormalized data model. You are trading off flexibility for efficiency.

考虑解决方案域建模的另一种方法是为每个用例创建一个表。举个例子，滑雪者和单板滑雪者喜欢使用他们的应用程序列出他们每个季节参观他们最喜欢的山脉的天数、他们乘坐的缆车次数以及天气状况。使用规范化，您可能会生成如下所示的逻辑数据模型，并创建实现该模型的表：

Another way of thinking about solution domain modeling is to create a table per use case. As an example, skiers and snowboarders love to use their apps to list how many days they have visited their favorite mountains each season, how many lifts they rode, and what the weather was like. Using normalization, you’d probably produce something like the following as a logical data model and create tables that implement the model:

SnowSportPerson = {ssp_id, ssp_name, 地址, 出生日期, ……….}
度假村 = {度假村 ID、度假村名称、位置、……}
访问 = {ssp_id、resort_id、日期、numLifts、垂直、…..}
天气 = {resort_id、日期、最高气温、最低气温、风、...}

SnowSportPerson = {ssp_id, ssp_name, address, dob, ……….}
Resort = {resort_id, resort_name, location, …..}
Visit = {ssp_id, resort_id, date, numLifts, vertical, …..}
Weather = {resort_id, date, maxtemp, mintemp, wind, …}

使用 SQL，可以很简单地JOIN生成特定人员的访问列表，如下所示：

Using SQL, it’s straightforward JOIN wizardry to generate a list of visits for a specific person that looks like the following:

• 摘要伊恩·戈顿天数： 2

• Summary Ian Gorton Number of days: 2

在 NoSQL 数据建模中，您创建一个数据模型，将查询所需的结果全部放在一个表中。如下所示，aVisitDay具有生成上面列表中每一行所需的所有数据项。您只需将结果集中的对象数量相加VisitDay即可计算单个人的天数。³

In NoSQL data modeling, you create a data model that has the results the query needs all together in a table. As shown in the following, a VisitDay has all the data items needed to generate each line in the list above. You just have to sum the number of VisitDay objects in the results set to calculate the number of days for a single person.³

VisitDay = {日期、度假村名称、ssp_id、ssp_name、numLifts、垂直、最高温度、最低温度、风}

VisitDay = {date, resort_name, ssp_id, ssp_name, numLifts, vertical, maxtemp, mintemp, wind}

SnowSportPerson 、Resort和Weather表将与原始模型保持不变。这意味着我们在您的逻辑表中存在重复的数据。在此示例中，这些表中的大部分数据都是一次性写入且永远不会更改（例如，特定日期的天气状况），因此复制只会占用更多磁盘空间，这不是现代系统中的主要问题。

The SnowSportPerson, Resort, and Weather tables would remain unchanged from your original model. This means we have duplicated data across your logical tables. In this example, most of the data in these tables is write-once and never changes (e.g., weather conditions for a particular day), so duplication just uses more disk space—not a major problem in modern systems.

但想象一下，如果度假村名称发生变化。它确实偶尔会发生。此更新必须检索该度假村的所有VisitDay条目并更新每个条目中的度假村名称。在非常大的数据库中，此更新可能需要几十秒或更长时间，但由于这是一项数据维护操作，因此可以在一个黑夜运行，以便第二天新名称神奇地出现在用户面前。

Imagine, though, if a resort name changes. It does actually happen occasionally. This update would have to retrieve all VisitDay entries for that resort and update the resort name in every entry. In a very large database, this update might take a few tens of seconds or more, but as it’s a data maintenance operation, it can be run one dark night so that the new name appears magically to users the next day.

所以你有它。如果您设计数据模型以根据主要用例有效处理请求，则不需要像连接这样的复杂操作。除此之外，数据分区和分发变得更加容易，并且好处开始大规模叠加。通常，读取速度更快而写入速度更慢，这是一种权衡。您还必须仔细考虑如何实现重复数据的更新并维护数据完整性。

So there you have it. If you design your data model to efficiently process requests based on major use cases, complex operations like joins are unnecessary. Add to this that it becomes easier to partition and distribute data and the benefits start to stack up at scale. The trade-offs are that, typically, reads are faster and writes are slower. You also have to think carefully about how to implement updates to duplicate data and maintain data integrity.

NoSQL 数据模型

NoSQL Data Models

如图10-6所示，NoSQL数据模型主要有四种，它们都比关系模型简单一些。

As illustrated in Figure 10-6, there are four main NoSQL data models, all of which are somewhat simpler than the relational model.

图 10-6。NoSQL 数据模型

从根本上来说，这些模型之间存在微妙的重叠。但忽略这些微妙之处，这四个是：

Fundamentally there are subtle overlaps between these models. But ignoring these subtleties, the four are:

核心价值

Key-value

键值（KV）数据库基本上是哈希图。数据库中的每个对象都有一个唯一的键，用于检索与该键关联的数据。对于数据库来说，与键关联的数据通常对数据库引擎是不透明的。它可以是字符串、JSON、图像或业务问题所需的任何其他内容。KV 数据库的示例包括Redis和Oracle NoSQL。

Key-value (KV) databases are basically a hash map. Every object in the database has a unique key that is used to retrieve data associated with that key. To the database, the data associated with the key is typically opaque to the database engine. It can be a string, JSON, image, or whatever else the business problem demands. Examples of KV databases include Redis and Oracle NoSQL.

文档

Document

文档数据库构建在 KV 模型上，数据库中的每个文档都需要唯一的键。与键关联的值对于数据库来说并不是不透明的。相反，它通常采用 JSON 进行编码，从而可以在查询中引用文档中的各个元素，并使数据库可以在文档字段上构建索引。文档通常被组织成类似于关系表的逻辑集合，但不要求集合中的所有文档都具有相同的格式。领先的文档数据库是MongoDB和Couchbase。

A document database builds on the KV model, again with each document in the database requiring a unique key. The value associated with the key is not opaque to the database. Rather it is encoded, typically in JSON, making it possible to reference individual elements in a document in queries and for the database to build indexes on document fields. Documents are usually organized into logical collections analogous to relational tables, but there is no requirement for all documents in the collection to have the same format. Leading document databases are MongoDB and Couchbase.

宽柱

Wide column

宽列数据库扩展通过组织与命名列中的键关联的数据来构建 KV 模型。它本质上是一个二维哈希图，使行中的列能够使用列名称进行唯一标识和排序。与文档数据库一样，集合中的每一行都可以有不同的列。Apache Cassandra和 Google Bigtable是宽列数据库的示例。

A wide column database extends the KV model by organizing data associated with a key in named columns. It’s essentially a two-dimensional hash map, enabling columns within a row to be uniquely identified and sorted using the column name. Like a document database, each row in a collection can have different columns. Apache Cassandra and Google Bigtable are examples of wide column databases.

图形

Graph

图表很好理解用于存储和查询高度关联的数据的数据结构。想想 Facebook 上的朋友，或者航空公司在机场之间飞行的航线。图将数据库对象之间的关系视为一等公民，因此可以有效地实现各种基于图的算法。从概念上讲，最接近关系数据库的突出例子是Neo4j和Amazon Neptune。

Graphs are well understood data structures for storing and querying highly connected data. Think of your friends on Facebook, or the routes flown by an airline between airports. Graphs treat relationships between database objects as first-class citizens, and hence enable a wide range of graph-based algorithms to be efficiently implemented. Conceptually closest to relational databases, prominent examples are Neo4j and Amazon Neptune.

无论数据模型如何，NoSQL 数据库通常被称为无模式数据库。与关系数据库不同，写入数据库的每个对象的格式不必预先定义。这使得可以轻松地发展数据对象格式，因为逻辑集合中的每个对象不需要具有相同的格式。

Regardless of data model, NoSQL databases are usually termed as schemaless databases. Unlike relational databases, the format of every object you write into the database does not have to be defined up front. This makes it possible to easily evolve data object formats as there is no need for every object in a logical collection to have the same format.

这种灵活性不可避免的代价是应用程序有责任发现它所读取的数据的结构。这需要将数据对象与元数据（基本上是字段名称）一起存储在数据库中，从而使结构发现成为可能。您经常会看到这两种方法，称为写入时架构（定义的架构）和读取时架构（无架构）。

The inevitable trade-off for this flexibility is that it becomes the responsibility of the application to discover the structure of the data it reads. This requires data objects to be stored in the database along with metadata (basically field names) that make structure discovery possible. You’ll often see these two approaches called schema-on-write (defined schema) and schema-on-read (schemaless).

db.skiers.find( {
   年龄：{$gt：16}，
   更新：{ $exists: false }}
）

db.skiers.find( {
   age: { $gt:  16},
   renew: { $exists: false }} 
)

MATCH (p:Person)-[rel:VISITED]->(c:Skiresort)
WHERE c.name = 'Mission Ridge'
返回电子邮件

MATCH (p:Person)-[rel:VISITED]->(c:Skiresort)
WHERE c.name = ‘Mission Ridge’
RETURN p.email

数据分布

Data Distribution

NoSQL 数据库的总体设计跨配备本地存储的分布式计算节点本机水平扩展。这是一个无共享架构，因为与我在 Oracle RAC 中描述的共享一切方法相反。由于没有共享状态，瓶颈和单点故障都被消除，⁵并且性能、可扩展性和可用性得到增强。这一规则有一个值得注意的例外，那就是图形数据库，正如我在下面的边栏中所描述的那样。

NoSQL databases are in general designed to natively scale horizontally across distributed compute nodes equipped with local storage. This is a shared nothing architecture, as opposed to the shared everything approach I described with Oracle RAC. With no shared state, bottlenecks and single points of failure are eliminated,⁵ and performance, scalability, and availability enhanced. There’s one notable exception to this rule, and that is graph databases, as I describe in the following sidebar.

分区，通常称为分片，需要一种将逻辑数据库集合中的数据对象分布到多个服务器节点的算法。理想情况下，分片算法应该在可用资源上均匀分布数据。也就是说，如果您有 1 亿个对象和 10 个相同的数据库服务器，则每个分片将有 1000 万个对象驻留在本地。

Partitioning, commonly known as sharding, requires an algorithm to distribute the data objects in a logical database collection across multiple server nodes. Ideally, a sharding algorithm should evenly distribute data across the available resources. Namely, if you have one hundred million objects and ten identical database servers, each shard will have ten million objects resident locally.

分片需要一个分片或分区键，用于将给定的数据对象分配到特定的分区。创建新对象时，分片键将该对象映射到驻留在服务器上的特定分区。当查询需要访问对象时，它会提供分片键，以便数据库引擎可以在其所在的服务器上找到该对象。如图 10-7所示。

Sharding requires a shard or partition key that is used to allocate a given data object to a specific partition. When a new object is created, the shard key maps the object to a specific partition that resides on a server. When a query needs to access an object, it supplies the shard key so the database engine can locate the object on the server it resides. This is illustrated in Figure 10-7.

图 10-7。数据分区

分片存在三种主要技术，所有分布式数据库都将实现以下一种或多种方法：

Three main techniques exist for sharding, and all distributed databases will implement one or more of these approaches:

散列键

Hash key

任何给定数据对象的分区被选择作为对分片键应用哈希函数的结果。然后哈希的结果被映射到分区。有两种主要方法可以做到这一点，使用模数方法或称为一致性哈希的算法。

The partition for any given data object is chosen as the result of applying a hash function to the shard key. The result of the hash is then mapped to a partition. There are two main ways of doing this, using a modulus approach or an algorithm known as consistent hashing.

以价值为基础

Value-based

分区的选择基于关于分片键的值。例如，您可能希望根据客户的居住国家/地区对客户数据进行分区。选择国家/地区字段作为分片键将确保居住在中国的客户的所有数据对象都驻留在同一分区中，所有芬兰客户都分配到同一分区中，依此类推。

The partition is chosen based on the value of the shard key. For example, you might want to partition your data on customers based on their country of residence. Choosing the country field as the shard key would ensure all data objects for customers who live in China reside in the same partition, all Finland customers are allocated to the same partition, and so on.

基于范围

Range-based

分区托管数据对象，其中片键位于片键值的特定范围内。例如，您可以使用邮政编码/邮政编码范围将位于同一地理区域的所有客户对象分配到同一分区。

Partitions host data objects where the shard key resides within a specific range of the shard key value. For example, you might use zip code/post code ranges to allocate all customer objects who reside in the same geographical area to the same partition.

分区可以通过添加处理和磁盘容量以及在这些额外资源之间分配数据来扩展数据库。但是，如果其中一个分区由于网络错误或磁盘崩溃而不可用，则无法访问数据库的一部分。

Partitioning makes it possible to scale out a database by adding processing and disk capacity and distributing data across these additional resources. However, if one of the partitions is unavailable due to a network error or disk crash, then a chunk of the database cannot be accessed.

解决这个可用性问题需要引入复制。每个分区中的数据对象通常被复制到两个或更多节点。如果一个节点不可用，应用程序可以通过访问副本之一继续执行。这种分区、复制的架构如图 10-8所示。每个分区都有三个副本，每个副本托管在不同的节点上。

Solving this availability problem requires the introduction of replication. The data objects in each partition are replicated to typically two or more nodes. If one node becomes unavailable, the application can continue to execute by accessing one of the replicas. This partitioned, replicated architecture is shown in Figure 10-8. Each partition has three replicas, with each replica hosted on a different node.

图 10-8。数据分区和复制，每个分区三个副本

复制增强了可用性和可扩展性。存储副本的附加资源可用于处理来自应用程序的读取和写入请求。

Replication enhances both availability and scalability. The additional resources that store replicas can be used to handle both read and write requests from applications.

然而，与分布式系统一样，有一个需要解决的复杂问题。当发生数据更新请求时，数据库需要更新所有副本。这确保了副本是一致的，并且所有客户端都将读取相同的值，无论它们访问哪个副本。

There is however, as always with distributed systems, a complication to address. When a data update request occurs, the database needs to update all replicas. This ensures the replicas are consistent and all clients will read the same value regardless of the replica they access.

有两种用于管理分布式数据库复制的基本体系结构。这些都是：

There are two basic architectures for managing distributed database replication. These are:

领导者-追随者

Leader-follower

一个副本被指定为领导者，它始终保存任何数据对象的最新值。所有写入都定向到领导者，领导者负责将更新传播到副本。追随者是只读副本。应用程序读取可以在追随者之间进行负载平衡，以扩展读取性能。

One replica is designated the leader and it always holds the latest value of any data object. All writes are directed to the leader, which is responsible for propagating updates to the replicas. The followers are read-only replicas. Application reads can be load balanced across the followers to scale out read performance.

群龙无首

Leaderless

任何副本都可以同时处理读取和更新。当更新发送到副本时，它成为该更新的请求协调器，并负责确保其他副本得到正确更新。由于写入可以由任何副本处理，因此对于写入量大的应用程序来说，无领导者方法往往更具可扩展性。

Any replica can handle both reads and updates. When an update is sent to a replica, it becomes the request coordinator for that update and is responsible for ensuring the other replicas get correctly updated. As writes can be handled by any replica, the leaderless approach tends to be more scalable for write-heavy applications.

事实证明，副本一致性是一个棘手的分布式系统问题。问题的核心在于关于如何以及何时将更新传播到副本以确保它们具有相同的值。不同的延迟以及网络和硬件故障等常见问题使得这变得非常重要。

Replica consistency turns out to be a thorny distributed systems issue. The core of the problem revolves around how and when updates are propagated to replicas to ensure they have the same values. The usual issues of varying latencies and network and hardware failures make this totally nontrivial.

如果数据库能够确保所有副本始终具有相同的值，则可以说它提供了强一致性，因为所有客户端访问都会为每个数据对象返回相同的值。这意味着客户端必须等到所有副本都被修改后才能确认更新成功。

If a database can ensure all replicas always have the same value, then it is said to provide strong consistency, as all client accesses will return the same value for every data object. This implies the client must wait until all replicas are modified before an update is acknowledged as successful.

相反，客户端可能只想等待一个副本更新，并相信数据库会尽快更新其他副本。这意味着您有一个时间窗口，副本不一致，并且读取可能会也可能不会返回最新值。允许副本不一致的数据库称为最终一致性。接下来的三章将详细讨论强一致性和最终一致性之间的权衡，以及设计选择如何影响可扩展性和可用性。

In contrast, a client may only want to wait for one replica to be updated, and trust the database to update the others as soon as it can. This means you have a window of time when replicas are inconsistent and reads may or may not return the latest value. Databases that allow replica inconsistency are known as eventually consistent. The trade-offs between strong and eventual consistency and how design choices affect scalability and availability are dealt with in detail in the next three chapters.

CAP定理

The CAP Theorem

Eric Brewer 著名的 CAP 定理⁶优雅地封装了在使用分布式数据库时实现副本一致性和可用性的选项。它描述了如果存在网络分区，即当网络丢弃或延迟数据库中节点之间发送的消息时，数据库系统可以做出的选择。

Eric Brewer’s famous CAP theorem⁶ elegantly encapsulates the options you have for replica consistency and availability when utilizing distributed databases. It describes the choices a database system has if there is a network partition, namely when the network drops or delays messages sent between the nodes in the database.

基本上，如果网络如果运行正确，系统就可以是一致的和可用的。如果发生网络分区，系统可以是一致的（CP）或可用的（AP）。

Basically, if the network is operating correctly, a system can be both consistent and available. If a network partition occurs, a system can be either consistent (CP) or available (AP).

出现这种情况是因为网络分区意味着数据库中的某些节点无法被其他节点访问——分区将数据库分成两组节点。如果发生更新并且更新的数据对象的副本驻留在分区的两侧，则数据库可以：

This situation arises because a network partition means some nodes in the database are not accessible to others—the partition splits the database into two groups of nodes. If an update occurs and the replicas for the updated data object reside on both sides of the partition, then the database can either:

• 返回错误，因为它无法确保副本一致性（CP）。

• Return an error as it cannot ensure replica consistency (CP).

• 将更新应用到可见副本子集 (AP)。这意味着在分区修复并且数据库可以使所有副本保持一致之前，会存在副本不一致的情况。在解决不一致之前，客户端可能会看到同一数据对象的不同值。

• Apply the update to the subset of replicas that are visible (AP). This means there is replica inconsistency until the partition heals and the database can make all replicas consistent. Until the inconsistency is resolved, clients may see different values for the same data object.

您将看到用于不同 NoSQL 数据库的 AP 或 CP 分类。它很有用，但并不完全有意义，因为正如我将在第 13 章中解释的那样，大多数数据库都可以调整配置参数以实现AP或CP以满足应用要求。

You’ll see the AP or CP categorization used for different NoSQL databases. It’s useful but not totally meaningful as most databases, as I’ll explain in Chapter 13, make it possible to tune configuration parameters to achieve AP or CP to meet application requirements.

总结和延伸阅读

Summary and Further Reading

随着系统规模的增长，数据库领域发生了一场革命。数据库必须存储大量数据，为全球分布的客户端提供快速的查询响应时间，并且 24/7 可用。这要求数据库技术变得分布式并采用新的数据模型，这些模型更适合现代应用程序所需的非结构化、不断变化的数据类型。

As the scale of systems has grown, a revolution has taken place in the database realm. Databases must store massive volumes of data, provide rapid query response times for globally distributed clients and be available 24/7. This has required database technologies to become distributed and adopt new data models that are more amenable to the unstructured, ever changing data types necessitated by modern applications.

在本章中，我解释了为什么关系数据库和 SQL 大规模时会出现问题。相比之下，NoSQL 数据库采用简单的数据模型，可以复制和分区，以支持海量数据集和请求量。一如既往，需要权衡。NoSQL数据库不支持SQL丰富的查询功能，给应用程序带来了较大的负担。分布式数据库设计者还需要了解 CAP 定理列举的一致性和可用性权衡。

In this chapter, I’ve explained why relational databases and SQL can become problematic at scale. In contrast, NoSQL databases adopt simple data models that can be replicated and partitioned to support massive data sets and request volumes. As always, there are trade-offs. NoSQL databases do not support the rich query features of SQL, placing a greater burden on the application. Distributed database designers also need to be aware of the consistency and availability trade-offs that are enumerated by the CAP theorem.

有了这些基础，接下来的三章重点讨论 CAP 定理推断的权衡的复杂性。我将解释在各种数据库中设计和实现的方法，以使应用程序能够平衡一致性、可用性和性能以满足其要求。

With these foundations, the following three chapters focus on the complexities of the trade-offs inferred by the CAP theorem. I’ll explain the approaches that have been devised and implemented in various databases to enable applications to balance consistency, availability, and performance to meet their requirements.

对于 NoSQL 数据库的精彩介绍，仍然很难超越Pramod Sadalage 和 Martin Fowler 所著的《NoSQL Distilled：A Brief Guide to the Emerging World of Polyglot Persistence》 （Addison-Wesley Professional，2013 年）。为了更广泛地涵盖数据库领域，包括 SQL 和 NoSQL， Andreas Meier 和 Michael Kaufmann撰写的《SQL 和 NoSQL 数据库：大数据管理的模型、语言、一致性选项和架构》（Springer，2019 年）非常值得一读。最后，如果本章激起了您深入了解数据库如何工作的兴趣，强烈推荐Alex Petrov 的《数据库内部原理》 （O'Reilly，2019 年）。

For an excellent introduction to NoSQL databases, it’s still hard to beat NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence by Pramod Sadalage and Martin Fowler (Addison-Wesley Professional, 2013). For broader coverage of the database landscape, including both SQL and NoSQL, SQL and NoSQL Databases: Models, Languages, Consistency Options and Architectures for Big Data Management by Andreas Meier and Michael Kaufmann (Springer, 2019) is well worth a read. Finally, if this chapter has whetted your appetite for learning about how databases work in depth, Alex Petrov’s Database Internals (O’Reilly, 2019) is highly recommended.